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ABSTRACT 


The MK 92 Mod 2 Fire Control System (FCS) is a complex, maintenance intensive 
shipboard weapon system found primarily aboard the Oliver Hazard Perry class guided missile 
frigates (FFG-7). This system, based on 1970's technology, frequently requires extensive 
troubleshooting and supplemental support from shore-based technical experts. A maintenance 
advisor expert system (MAES) is bring developed by the Port Hueneme Division of the Naval 
Surface Warfare Center (NSWC-PHD) and the Naval Postgraduate School (NPS) to assist 
the Fire Control technicians aboard ship to better isolate faults in the MK 92 Mod 2 FCS. 

This thesis furthers the efforts of the project at NPS by investigating key 
implementation issues that will affect the deployment of the initial version of MAES to the 
fleet. Additional deployment issues addressed in the thesis include incorporating lessons 
learned from deploying other ejq)ert systems in the armed forces, gaining support from 
individual chains of command, training MAES users effectively, involving MAES users in the 
implementation process, and changing hardware implementation issues. A training plan, 
implementation plan, and updated MAES user’s manual for the initial deployment are 


included. 
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I. INTRODUCTION 


A developmait project initialized by the Naval Surface Warfare Cento*, Port Hueneme 
4C00 (NSWC-PHD), supported by the Naval Postgraduate School (NFS), and sponsored by 
the Naval Sea Systems Command (NAVSEA), the Mark 92 (MK 92) Mod 2 Fire Control 
System (FCS) Maintenance Advisor Expert System (MAES) project was initiated in 1992. 
It will serve as a tool to aid shipboard Fire Control technicians in diagnosing and 
troubleshooting faults that occur in the MK 92 FCS. MAES has other benefits that include: 

• Reduced mean-time-to-repair (MTTR) which results in improved operational 
readiness. 

• Significant cost savings from a reduction in the number of No Fault Evident (NFE) 
circuit cards replaced. 

• Reduced reliance on outside technical support activities. 

• A capability to more efifectively train junior fire control technicians in the 
troubleshooting of difificult system faults in general. 

• Proving the value of expert systems which may lead to the development of expert 
systems for other complex systems such as the MK86 GFCS or the SQQ-28 
SONAR processor. 

Initial prototype versions of MAES have been developed and tested. Copies of the initial 
version have been distributed for evaluation to: Fleet Training Center, San Diego (FTC-SD), 
USS Sides (FFG-14), USS Wadsworth (FFG-19), and Fleet Combat Training Center (FCTC), 
Norfolk. In Jialy 1995, under the sponsorship of Commander Naval Surface Forces Atlantic 
(COMNAVSURFLANT), a Navy Sdentific Assistance Program (NSAP) grant was approved 
that would support the implementation and evaluation of the system aboard 
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COMNAVSURFLANT sJiips. The next step will be the deployment of the initial production 
version onboard six East coast based fiigates. An effective implementation plan is essential 
in the transition of the system from a prototype to a deployable system and is the focus of this 
thesis. 

A. OBJECTIVES 

This research is directed toward the successfiil implementation of the initial 
deployment version of MAES. Proper management of the deployment process is critical to 
the effective allocation of resources. The thesis has three main objectives. The first is to 
detamine the potential implementation issues that may affect the initial deployment of MAES. 
The second is to apply the applicable implementation issues to the deployment of MAES. 
The third objective is the development of an implementation plan, training plan, and a revised 
MAES user’s manual that address the implementation issues. 

B. RESEARCH QUESTIONS 

This thesis attempts to answer the following research questions: 

1. Primary Research Question 

• What are the potential implementation issues for the initial fielding of MAES? 

2. Subsidiary Research Questions 

• How can the identified implementation issues be addressed for the deployment of 
the initial version of MAES? 

• What are the end-user training concepts to be applied to the formulation of a 
MAES training plan for shipboard technicians? 
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• What are the hardware implementation issues associated with the initial 
deployment of MAES? 

• What are the lessons learned from the implementation of other expert systems for 
weapons ^sterns within DOD and to what degree are they applicable to the 
MAES deployment? 

C. SCOPE 

The scope of the theas is limited to examining the initial deployment implementation 
issues and development of: (1) An implementation plan for the initial deployment of MAES; 
(2) A training plan for shipbo^d technicians to be used during the initial deployment; and (3) 
An updated User’s Manual. 

D. METHODOLOGY 

The research methodology for this thesis includes a thorough literature review of 
software applications implementation with an emphasis on expert system implementation. In 
addition, interviews with personnel from other DOD activities that have implemented expert 
syst«ns were conducted. Addressing the implementation issues for the actual deployment of 
MAES follows the implementation framework developed by Prerau (1990). 

E. THESIS ORGANIZATION 

This thesis consists of four chapters and five appendices. The following is a brief 
description of the contents of each chapter. 

Chapter n provides the reader with the issues associated with implementing a software 
application. The development model developed by Prerau (1990) is presented and is being 
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used to implement MAES. The Lucas-Ginzberg (1990) model for implementation is 
introduced to determine the implementation fectors which need to be considered. Differences 
in expert system implementation are pointed out. 

Chapter m establishes the implementation issues that can be influenced by the 
deployment team. Different methods of influencing implementation issues are explored. 

Chapter IV evaluates the implementation issues determined in Chapter n and applies 
them to the implementation of the initial version of MAES. Lessons learned fi'om the 
deployment of other DOD diagnostic expert systems are discussed. 

Chapter V provides a thorough review of the duties and responsibilities of tiie 
deployment team. Hardware implementation issues are discussed as they apply to the 
deployment of MAES. The chapter also discusses the contents and purpose of an 
implementation plan. 

In Chapter VL, the research is summarized, and conclusions are drawn. Finally, 
recommendations are made for the initial deployment and for further research. 

Appendix A is an implementation plan for the initial deployment of MAES. Appendix 
B is a basic training plan and includes a training evaluation form to be used by the deployment 
team. Appendix C is an updated version of the MAES user’s manual and Appendix D is a 
MAES User Survey Form. 
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II. IMPLEMENTATION ISSUES RESEARCH FOR TRADITIONAL 
AND EXPERT SYSTEMS SOFTWARE 

This chapter discusses the potential implementation issues involved in fielding 
software applications and addresses the primary research question, “What are the potential 
implementation issues for the initial fielding of MAES?” Section A reviews research on 
implementation issues. Section B explains the foundations of a successful implementation 
using the Prerau (1990) implementation model. Section C examines implementation fi-om a 
user and management perspective following the Lucas-Ginzberg (1990) model. Sections D 
and E discuss how expert systems differ from traditional information systems and how expert 
systems can be categorized. Sections F and G examine variables that need to be considered 
when implementing an expert system and the relative importance of implementation factors 
depending on the category of expert ^stem. Section H provides a summary of the chapter. 
A. SUMMARY OF IMPLEMENTATION ISSUES RESEARCH 

A great deal of research has been done on the different factors that can contribute 
towards a successful implementation of an information system. This section summarizes the 
research on implementation issues that was most applicable to this thesis. Multinovich and 
Vlahovich (1984) identified thirteen steps which are necessary in a successfiil implementation 
strategy. They determined that the steps are interrelated and that some may be done 
concurrently. Leonard-Barton and Deschamps (1987) concentrated on assessing the effect 
that upper management can have in the implementation of new technology. Lucas and 
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Ginzberg (1991) devoted a whole book to the development and testing of a structural model 
for information systems implementation. 

Research has also been conducted in the area of implementing expert systems. Prerau 

authored a book that provides a step-by-step approach to developing and managin g expert 

systems (Prerau, 1991). Bradley and Hauser (1995) provide a framework for ejq)ert system 

implementation based on categorizing the type of expert system that is being implemented. 

B. , PRERAU’S METHODOLOGY FOR IMPLEMENTING EXPERT SYSTEM 
SOFTWARE APPLICATIONS 

While the development of any application has unique aspects, Prerau maintains that 
there are three basic phases in the evolution of a system that can be considered generic: 
(Prerau, 1990) 

• The Initial Phase composed of project start-up, selection of the domain and 
selection of the development environment. 

• The Core Development Phase consisting of the development of a feasibility 
prototype system and foil prototype system. 

• The Final Development and Deployment Phase composed of the development of 
a production system, system deployment, and system operation and maintenance. 

According to Prerau, ensuring that an application is developed using the above basic phases 

as a foundation will better ensure the successful development and deployment of a new 

system. The reader may wish to review Prerau’s discussion of the Initial and Core 

Development phases. This section will focus on the third phase - the final development and 

deployment phase of a project. 
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In the final phases of software development, Prerau states that the fina l production 
system is built and the system is deployed. The program then transitions to a maintenance 
phase, with program and knowledge bugs fixed and possibly new information and features 
added. 


1. Development of a Production System 

According to Prerau, the decision to develop a production-quality version of an 
application fi-om the full prototype system is based on several factors: the results of the field 
tests of the prototype system; feedback fi-om outside experts, potential users, and 
management; and estimates of the cost versus benefits of the final system. D'the dedsion is 
made to proceed, then the development of the production system is initiated. In this phase, 
the fiill prototype is made into a program that is robust, reliable, and can be fielded. Then the 
system is deployed and put into long-term operation. To produce the deployment system, 
Prerau suggests several issues be explored: (Prerau, 1990) 

• Possible representation and re-implementation: There may be benefits in doing a 
redesign and re-implementation when beginning work on the deployment system. 
For example, when it is decided that the final production system should be 
rewritten to allow it to be run on new hardware or with a new software tool, the 
additional cost of redesigning the representation and implementation mi ght be 
lower. 

• Deployment hardware: The deployment hardware does not necessarily have to be 
the same as the development hardware, although sometimes the hardware for 
development was selected with the deployment in mind. Some of the fectors 
involved in hardware selection include: reducing hardware cost, making progr am 
operation and maintenance easier, using a hardware vendor that provides better 
support, and putting the application on hardware that can be used for other 
purposes. 
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• Deployment software: The deployment software tool does not have to be the same 
as Ae development software tool. If the deployment software is different, the cost 
of converting firom the development software will have to be considered. 

• Input/Output formats and mechanisms: The formats and mechanisms for user 
inputs and to deliver output to users may have to be reconsidered. Comments 
fi-om the users involved in the field tests and from domain experts may aid in the 
design of good input/output formats. 

• Testing: The production system should be tested and evaluated. Validation tests 
should determine that the system effectively solves the problem for which it was 
built, at an acceptable level of ejq)ertise. Verification tests should ensure that the 
expert knowledge acquired has been correctly implemented, that the system is 
operating accurately to the extent required and is as free as possible of 
programming errors. 

• Documentation: The documentation that will accompany the system must be 
designed and written. It might consist of printed manuals, on-line help, or both. 
There may be levels of documentation for system maintainers, system operators, 
and system users. 

2. System Deployment 

When the production system is completed and fully tested, the system can be deployed 
(Prerau, 1990). There are several factors that Prerau (1990) recommends analyzing before 
deployment: 

• Mode of deployment: There are several options. The final system could be 
delivered to users as a stand-alone system; it could be operated as a separate entity 
but integrated into the user’s environment; or it could be embedded into another 
system. The users of the system may be responsible for operating it, they may 
have responsibility for both operating and maintaining it, or they may utilize the 
system as a service. 

• System introduction: If the users have been clearly identified and involved 
throughout the development, introducing the system into the working environment 
may not pose many problems. Users to whom the system is new may have to be 
convinced to change their present mode of operation. Introduction of a new 
system will change the status quo and may arouse the fears associated with 
automation. Beyond these standard problems, expert systems have an additional 
aspect that may cause problems and that is inherent to the technology: expert 
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systems sometimes make mistakes. Often users can accept incorrect results more 
ea^y from a human ejq)ert than from a computer program. When a system makes 
mistakes there may be problems getting users to trust the system. 

• User training; The methods to be used for training system operators and users 
must be decided. Training may include formal courses, training manuals, and/or 
on-line tutorials. 

• Documentation: Any documentation for system operators, maintainers, or users 
not already written should be developed. 

Although these areas are directly related to deployment, many of the issues can be examined 
and choices made vdiile the production system is under development and testing. Many issues 
may be resolved earlier in the project. A project should have a deployment plan specifying 
the selected deployment mode, hardware, software, pricing, and so forth. It should specify 
the sequence and timing of events that will be followed to bring about the initial system 
deployment. (Prerau, 1990) 

3. System Operation and Maintenance 

According to Prerau, a long term maintenance group should be formed and trained. 
Maintenance not only includes fixing problems found during system operation, but also 
revising internal data and knovdedge that changes over time. Since maintenance will require 
not just changes to the implementation of knowledge but also changes to the knowledge itself, 
it is mandatory that a domain expert be involved in the maintenance process, even on a 
consulting basis only. A decision must be made whether to have centralized maintenance, 
where program patches and revisions come from a single source, or to have a more 
distributed maintenance plan. The latter approach will result in non-standard versions of the 
system and may not be desirable. However, it does allow the customization of the software 
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to fit the circumstances of each site. It may become necessary, during the lifetime of an 
expert system, to make major changes to the software to expand its scope or add new 
features and capabilities. System revision can be done by the system maintenance group, but 
if the expansion is large enough, it may be necessaiy to use an independent development 
group. (Prerau, 1990) 

C. IMPLEMENTING A SOFTWARE APPLICATION 

Deployment of a new software application entails bringing such a ^stem into 
operational use by turning it over to the end user (Multinovich and Vlahovich, 1984). 
Ultimately, it is end users who determine the success of a system through their use of the 
system. The deployment process can be considered a critical success factor in whether a 
sof^are application is accepted by the end users. Lucas and Ginzberg (1990) developed an 
implementation model based on previous research on implementation issues. They conclude 
that the most consistent relationships with software success or Mure have been demonstrated 
to be three major issues: management support, user involvement, and conduct of the 
implementation process itself 

1. The Manager Model for Implementation 

The Lucas and Ginzberg model consists of two separable submodels, the user model 
and the manager model. The manager model in Figure 2-1 consists of the following variables: 
(Lucas and Ginzberg, 1990) 

• Manager acceptance: This variable is a measure of the extent to which a manager 
wants a particular system to be implemented. 
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Figure 2-1. The Lucas-Ginzbei^ manager sub-i 
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• Manager knowledge of system: A measure of how well a manager understands the 
system to be implemented. Better understanding of a system’s design and 
capabilities leads directly to increased acceptance. 

• Manager assessment of system and support: Measures the manager’s evaluation 
of the quality of the system and its supporting mechanism. Favorable evaluations 
should result in increased acceptance. 

• Manager belief in system concept: The extent to which a manager believes in the 
underlying concept or approach behind a system and the ability of that approach 
to solve the organization’s decision problems. Stronger belief in the ^stem 
concept will result in greater incentive for the manager to become involved in 
system development and leam about the system. 

• Top management support: This variable measures the level of support exhibited 
by top management in an organization for the use of a particular system. Greater 
top management support should result in managers being more willing to become 
involved in system development and having greater belief in the system concept. 

According to Lucas and Ginzberg, an implementation process begins with management 

initiation and acceptance, so the management issues are usually addressed first. A ^stem that 

is accepted by management still stands a chance of failure if the users do not accept it. But 

ensuring that the above variables are addressed and optimized increases the probability of a 

successful deployment. (Lucas and Ginzberg, 1990) 

2. The User Model for Implementation 

One important objective of a system is to improve the user’s job performance. 
Therefore the Lucas and Ginzberg user model begins with the user’s personal stake in the 
system and ends with satisfaction and performance. The user model in Figure 2-2 consists 
of the following variables: (Lucas and Ginzberg, 1990) 

• User Acceptance: This variable measures the potential user’s predisposition to 
personally use a specific system. It is a measure of behavioral intention that will 
be reflected in actual use. 
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Figure 2-2. The Lucas-Ginzberg user sub-model. 
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• User knovdedge of system: This variable measures how much the user understands 
the functioning of a system. Better knowledge of a system’s design and 
capabilities leads to increased acceptance. 

• User assessment of system and support: Measures the user’s evaluation of the 
system and its supporting mechanisms. 

• System characteristics: This variable represents the features abd capabilities of the 
system. Examples of these characteristics may be ease of use or the fit between 
system capabilities and the demands of the user’s job. Easy to use systems which 
meet the user’s needs are more likely to be accepted. 

• User-researcher involvement: Indicates the degree of interaction between a user 
and the system deagner. Greater involvement leads to greater user knowledge of 
the system capabilities. 

• User perception of management support: One of the key determinants of the user’s 
acceptance is the manager’s acceptance or support of the system. It is the user’s 
perception of management support, not an actual measurement. 

• User knowledge of system purpose and use: Without knowledge of system 
purpose or use, the user will be unable to assess the importance of the system. 

• Problan urgency: The greater the user’s perception of problem urgency, the more 
important a system addressing that problem and the greater the user’s stake in that 
problem. 

• Organizational support: The degree to which the organization provides the 
environment and facilities to make access to and use of the system easy. 

A ^stem deployment can be viewed as an ongoing process. It continues as long as 
new users are bang introduced to the system. The concept of introducing a system into the 
existing user operation without major upheavals will require following both Prerau’s ( 1990 ) 
implementation methodology, while at the same time paying close attention to the controllable 
variables in the Lucas-Ginzberg implementation model of introducing a system to the user. 
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Prerau states that for a successful deployment of an expert system, there must be 
clearly defined goals for the fielding (Prerau, 1990). From the discussion of the previous 
sections, these goals can be summarized as follows: 

• Obtaining management support. (Prerau, 1990 and Lucas-Ginzberg, 1990) 

• Involving the users in the development. (Prerau, 1990 and Lucas-Ginzberg, 1990) 

• Involwng the users in the evaluation of the prototype. (Prerau, 1990) 

• Proper training of the users. (Prerau, 1990 and Lucas-Ginzberg, 1990) 

• Establishing effective lines of communication between the users and the 
maintenance team. (Prerau, 1990) 

• Conducting a post-deployment evaluation. (Multinovich and Vlahovich, 1984) 
The latter sections in this chapter examine the specifics of meeting the goals for successMy 
deploying a software application. The following section discusses the differences between 
an expert system and a traditional information system. 

D. DIFFERENCES BETWEEN AN EXPERT SYSTEM AND A TRADITIONAL 

INFORMATION SYSTEM 

Expert systems differ fi-om traditional management information systems in a number 
of ways. These differences include the type of problem-solving approach, the source and 
acquisition of system specifications, and the physical design of the system. First, expert 
systems employ a heuristic approach rather than the algorithmic approach used by traditional 
information systems. Second, expert systems are designed to provide a solution to a problem 
based solely on the knowledge of one or more experts. This can limit the degree of user 
involvement in the design process because the source of the system’s knowledge comes 
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from an expert. The reliance on the talent and skill of a knowledge engineer is also a major 
difference from traditional system requirements. (Bradley and Hauser, 1995) 

Another difference in the development process lies in who physically controls the 
development. Traditional information system development has been handled by the MIS 
department. Expert system development, however, is different enough that management is 
often uncertain as to the level of involvement of the MIS department in the development 
process. Expert systems are often developed to diagnose mechanical problems in complex 
machinery where technicians, such as industrial engineers, are used in the development 
process since they understand the problems and are comfortable with computers as tools. 
This approach can result in a bypass of traditional system development life cycle requirements. 
Surveys of expert systems found that more than 50% were developed outside the MTS 
department. (Bradley and Hauser, 1995). 

Expert systems possess unique characteristics that distinguish them from traditional 
management information systems. Expert systems also vary from one another. The following 
subsections discuss the differaices between expert systems and how the previously discussed 
variables from the Lucas-Ginzberg model and the Prerau development process affect expert 
system development. 

E. CATEGORIES OF EXPERT SYSTEMS 

Expert systems possess distinct attributes that differentiate them from one another. 
If such differences exist between expert systems, then it is possible that a single 
implementation plan would not be appropriate for all expert systems. Therefore, the first step 
in expert system implementation planning should include categorizing the system according 
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to its attributes and developing an implementation plan based on those attributes. Expert 
systems can be broken down into four categories based on the complexity of the knowledge 
they contain and the level of computing technology used to run them (Meyer and Curley, 
1991). Meyer and Curley developed a classification scheme (Figure 2-3) that views expert 
systems along these two attributes. 
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Within the expert system classification scheme, a system can be classified into one of four 
gropps: (Bradley and Hauser, 1995) 

• Category I: Low to mid knowledge complexity; Low to mid technical complexity. 
These ^sterns are developed in a problem domain that contains highly structured 
knowledge that can be ejq)ressed in IF-THEN rules. They enhance the 
productivity of a very small group of users who focus on a narrow problem, or a 
large group of users who make simple, repetitive decisions. These systems are 
often developed as a PC based applications using a relatively inexpensive expert 
system shell. 

• Category 11: Mid to high knowledge complexity; Low to mid technological 
complexity. This type of expert system is also typically implemented using an 
expert system shell. These systems address problems in more complex problem 
domains and require elaborate reasoning in specialized fields. The knowledge 
required for these systems is much more complex and usually involves true 
expertise instead of commonly used procedures. 

• Category DI: Low to mid knowledge complexity; Md to high technical 
complexity. These ejq)ert systems address problems in highly structured, simple 
domains. The technology required to implement the system involves special user 
interfeces, complex database interactions, and usually a large programming effort. 
These expert systems require the technical knowledge and skill of a computer 
scientist for development. 

• Category IV: Md to high knowledge complexity; Md to high technical 
complexity. This type of expert system is a combination of types n and HI. The 
combination of complex knowledge and advanced technologies requires a 
development team of trained knowledge engineers and computer scientists. The 
development effort for these ^sterns usually includes complex progr amming , data 
base management, and systems integration as well as an extensive knowledge 
acquisition effort. 

Knowing that expert systems differ not only fi-om traditional management information systems 
but also from each other can prove valuable in planning an implementation. 

F. VARIABLES THAT AFFECT EXPERT SYSTEM IMPLEMENTATION 

In addition to the implementation factors previously addressed in this chapter, the 
unique nature of expert systems carry with them additional considerations for implementation. 
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Ejq)ert systems often require anployees to cease their current pattern of behavior, learn a new 
pattern of behavior that incorporates the innovation, and then persist in the new behavior. 
Implementation of an expert system involves modifying an individual’s behavior to include 
use of the erqrot system. Bradley and Hauser (1995) identified a number of factors that can 
influence user acceptance of an expert system. The factors are organized into three mmn 
groups: iimovation characteristics, organizational influences, and individual differences. 

1. Innovation Characteristics 

In expert system implementation, user attitudes may be influenced by attitudes 
towards computers in general. Prior expectations about the expert system have also been 
shown to be an important factor in successful implementation. Ginzberg (1981), found 
evidence that when users hold unrealistic prior expectations about a system, they are more 
resistant to change and the implementation is more Ukely to fail. 

2. Organizational Influences 

Leonard-Barton found that organizational influences impact on user acceptance 
(Leonard-Burton, 1987). Later sections of this chapter discusses a number of studies that 
have concentrated on the effects of organizational influence on implementation success. 
Organizational influence factors include, but are not limited to, user participation in 
development, rewards for using the system, user training, and formal and informal 
management support. (Bradley and Hauser, 1995) 

3. Individual Differences 

Individual differences, such as user education and experience, have been shown to 
have a significant effect on implementation. A key factor in the personal characteristics of the 
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user is their attitude towards change. The extent to which a user perceives a change to then- 
job oTvironment will often effect the amount of resistance they exhibit. The effect of personal 
characteristics on implementation is dependent upon the perceived change caused by the 
expert system. (Bradley and Hauser, 1995) 

G. RELATIVE IMPORTANCE OF EXPERT SYSTEM IMPLEMENTATION 

FACTORS 

Bradley and Hauser (1995) suggest that any ejqpert system implemaitation plan should 
include a careful consideration of fectors in the three categories: innovation characteristics, 
organnational differences, and individual differences. The relative significance of each factor, 
however, will vary as a result of the type of expert system being implemented. Table 2-1 
represents each factor with a High, Medium, or Low value to indicate the degree of emphasis 
the fector should have in an implementation plan based on the classification of expert system. 
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Factor 

Expert System Classification 

T n m TV 

Innovation Characteristics 





User perception of computing technology 

H 

H 

H 

H 

User perception of expert systems 

L 

M 

M 

H 

Organizational influences 





User involvement in expert system design 

H 

H 

H 

H 

User involvement in expert system implementation 

H 

H 

H 

H 

Reward structure 

L 

M 

M 

H 

Training 

L 

M 

M 

H 

Top management support 

L 

M 

M 

H 

Supervisor support 

H 

H 

H 

H 

Advocates 

L 

M 

M 

H 

Consulting aids 

L 

M 

M 

M 

Individual Differences 





User perception of change 

H 

H 

H 

H 


L = low emphasis in the implementation plan 
M= medium emphasis in the implementation plan 
H = high emphasis in the implementation plan 


Table 2-1. Relative Importance of Expert System Implementation Factors. 
(Bradley and Hauser, 1995) 


The above framework suggests that a single implementation methodology may not be 
appropriate for all expert systems. Expert system developers should categorize the expert 
system early in the analysis of the system and develop an implementation plan that emphasizes 
what is most ^propriate for that system. In a resource constrained project, this method can 
also be used to determine where resources should optimally be concentrated to better ensure 
successful system implementation. (Bradley and Hauser, 1995) 
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H. SUMMARY 


In this chapter the Prerau (1990) methodology for implementing software applications 
was examined as a proven, practical approach for developing and managing the 
implementation of expert systems. Additionally, the Lucas-Ginzberg (1990) model for 
implementation was examined as it provides both the user’s and management’s perspective 
towards implementation. Finally, implementation issues that are specific to expert systems 
were addressed and found to be dependent on the type of expert system being implemented. 
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III. IMPORTANT IMPLEMENTATION ISSUES 

This chapter provides the background information needed to address the primary and 
subsidiary research questions listed in Chapter I. Sections A, B, and C examine methods of 
obtaining management support, involving the user in the development, and training the users, 
respectively. Sections D and E discuss establishing effective communications and conducting 
a post-implementation evaluation. Section F analyzes hardware concerns and Section G 
provides a summary of the chapter. 

A. MANAGEMENT SUPPORT 

This section examines the research conducted on the importance and methods of 
obtaining management support and also methods of overcoming the resistance of management 
to adopt a new system. 

1. The Importance of Obtaining Management Support 

Since manag ers and users interact on a regular basis, the manager’s attitude toward 
a system is an important fector for deployment success. Management support has been 
identified by a number of researchers as a key factor of deployment success (e.g. Ginzberg 
(1981), Multinovich and Vlahovich (1984),Leonard-Barton and Deschamps (1988), Bouldin, 
(1989), Prerau, (1990), Lucas-Ginzberg (1990)). Their major findings were: 

• Management commitment and support are critical and essential for successfiil 
implementation of MIS/DSS. (Ginzberg, 1981) 

• Capital resources required for system implementation increase significantly, and 
managptmfint must approve these expenditures. (Multinovich and Vlahovich, 1984) 
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• Subordinates are much more prone to act when their superiors are interested in 
the outcome, ^ultinovich and Vlahovich, 1984) 

• Perceptions of managerial behavior supporting or directly urging use of an 
innovation are positively related to use. (Leonard-Barton and Deschamps, 1988) 

• Obtaining upper management approval is the point in the implementation beyond 
which you cannot proceed without their consent. (Bouldin, 1989) 

• Obtaining management approval is one of the key steps in the initial phases of an 
implementation plan. (Prerau, 1990) 

• One of the key determinants of the user’s personal stake is his or her manager’s 
acceptance or support of the system. (Lucas-Ginzberg, 1990) 

For most software implementations, this will be a critically important step of the deployment 

process. Management support is important because it: (Multinovich and Vlahovich, 1984) 

• Provides adequate resources. 

• Provides psychological support. 

• Creates a climate that acknowledges a problem exists. 

In the Navy, a fector which is very applicable to the deployment process is that subordinates 
are much more prone to act when their superiors are interested in the outcome ^Multinovich 
and Vlahovich, 1984). 

2. Methods of Gaming Support and Overcoming Resistance 

Multinovich and Phatak (1981) conducted research to discover methods of obtaining 
management support. They propose several methods to gain support fi-om upper 
management: 

• “Sell” the system to management. Management must be apprized of the benefits 
of the ^stem and convinced that it would be to their advantage to support and 
implement the system. Implementation of any new technology is in essence a 
marketing task (Leonard-Barton, 1987). 
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• Create the illusion of support or generate support by sending copies of reviews to 
top management, particularly action memos. Others not aware of middle 
managemait’s support may be led to believe top management supports the system 
and will in turn support it. 

• Ensure that top management reads the memos. By reading the memos, they may 
change their opinion and support the system. 

Bouldin also developed some methods for overcoming management resistance: (Bouldin, 
1989) 


• Conduct a marketing blitz to ensure that management has some basic level of 
understanding about the software and its usefulness. Some methods to achieve this 
heightened awareness are: Utilize email, show videotapes of the software in action, 
give formal and ad hoc demonstrations and presentations, and utilize your informal 
network. 

• Conduct demonstrations to management to show off the capabilities and 
advantages of the new software. 

• Gain the support of technical experts whose opinion is trusted by management and 
have them available for testimony during presentations. 

• Publicize the results of a fevorable cost-benefit analysis. 

Obtaining the tadt support of management is not enough. Management must indicate 
through their words and actions that they support the system (Lucas and Ginzberg, 1990). 
Actions that show support include: (Lucas and Ginzberg, 1990) 

• Attending meetings with the users about the system. 

• Providing time for the users to train on the system. 

• Becoming knowledgeable about the basics of the system. 

• Personally using the system. 
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In order for management to actually participate in the above actions, they must first be 
convinced of the benefits of the system. Benefits that will likely appeal to management 
include: (Multinovich and Phatak, 1981) 

• Saving money: May require a cost-benefit analysis. 

• Improving Quality of Life: An increase in fi'ee time may result because of a 
possible decrease in the workload. 

• Using the system as a training tool: An intangible benefit that may be hard to 
quantify. 

Benefits such as those listed above would go a long way in convincing management that it 
would be to their advantage to support and implement the system. Management support is 
so necessary that Multinovich and Vlahovich (1984) recommend aborting any fiuther 
development of a system if management support is lacking. Continuing the development in 
such cases would be a waste of time and money. 

3. Cost-Benefit Analysis as a Method of Obtaining Management Support 

The purpose of a cost-benefit analyas is to assess as accurately as possible the benefits 
and costs associated with implementing a software application. The analysis of benefits must 
include determining both tangible and intangible benefits. Tangible benefits include things that 
can be measured such as reduction in staff size, reduced maintenance costs, and increased 
handling capacity. Intan^ble benefits include things that are difficult to measure such as 
improving employee morale and increased training opportunities. (Powell, 1993) 

After the analysis has been completed, the results must be analyzed carefully. 
Although management may not ever see the exact figures, it is important to be conservative. 
The main reason that an analysis is done is to enable a person to address whether or not a new 
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system will be a substantial improvement. The quantification will bring some objectivity to 
an otherwise subjective issue. Management will feel more comfortable if the advocate feels 
confident in promoting a new system. If the estimates indicate a promising area for 
productivity improvement, the figures can be used as a basis for obtaining the support of 
management. (Bouldin, 1989) 

B. USER INVOLVEMENT IN THE DEVELOPMENT 

Research has shown that users should actively be involved in the development of a 
system through its life cycle (Ginzberg, (1978), King (1979), Madni (1988), and Lucas- 
Ginzberg (1990)). The key findings fi’om their research include: 

• A new system should satisfy a particular user’s requirements without adversely 
affecting the information requirements of other users in the system. (King, 1979) 

• Greater involvement of the user in the development should lead to greater user 
knowledge of the system’s capabilities. (Lucas-Ginzberg, 1990) 

• Rq)orts should be made available to users if pertinent. This helps them to feel they 
are a part of the implementation and encourages involvement. (Ginzberg, 1978) 

• The single most important human factors issue is understanding the requirements 
of the intended user. (Madni, 1988) 

User participation must not be token nor given lip service. It must be meaningful and 
genuine. Enlisting the aid of the users in the development of software is prudent since the 
users often know much more than can be learned through interviews or surveys. Users have 
often been through solutions that did not work and may have observed solutions that could 
have worked if the users had had some type of input to the design (Multinovich and 
Vlahovich, 1984). 
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The knowledge by a user that his suggestion was implemented in the design of an MIS 
stimulates his enthusiasm for the system. Users should therefore be made to feel that they are 
contributing to a system, no matter how small the suggestion. This not only helps the user 
develop realistic ejq)ectations about the system’s capabilities, but it can also result in positive 
feelings of self-worth and ownership that decrease user resistance to change and commit the 
users to the system. (Bradley and Hauser, 1995) 

It must be understood that user participation is not a total solution but only one part 
of a successful implementation. There are concerns which should be kept in mind. Users are 
not professional developers, and, without professional assistance, they tend to miss 
opportunities for improvement and the applications of appropriate new information 
technologies. Since users do not generally think in terms of the whole process, their redesign 
ideas may be limited or flawed. Strong proponents for users participating in system design 
increasingly warn that it can lead to a form of “incrementalism,” rather than any appreciable 
performance improvements. Even when users have good ideas, there still lies the 
communication barrier that can exist between the users and the developers. Developers often 
have dfficulty understanding why a user desires a certain feature especially if it conflicts with 
the developer’s concept of how the system can or should be designed. Greater participation 
by users is not a total solution, but that is not saying that their participation has no value. 
Participation is necessary, but not sufficient for good system design concepts. Developers’ 
unaided perceptions about users’ jobs are often wrong. In conclusion, users and developers 
must design a system together. (Markus and Keil, 1994) 
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C. USER TRAINING 


This section discusses the importance and contributions of user training towards a 
successful implementation. Determining training needs and examination of effective training 
methods will be addressed as factors in formulating an effective training plan. Finally, 
concluaons will be drawn based on available information on user training as it contributes to 
the success of software implementation. 

1. The Importance of Training 

Although user interfeces are designed to be easy to use and prompt the user through 
system functions, training is still required. This is especially so if the user has minimal 
experience using computers or expert systems. A successful training program will convince 
any skeptics that the system is both crucial and necessary. A good initial impression of an 
application can be established by means of effective training. Properly trained users will have 
a superior comprehenaon and awareness of the project. Without proper training, the success 
of an application can be at risk because training ^ves the end-users the required skills and 
knowledge to reach anticipated levels of productivity. Training also produces greater 
confidence. This translates into the end-user referring to the computer as “my computed’ 
instead of “the computer.” As a result, trained users will use an application more efficiently 
and effectively, decreasing their dependence on support. Computers and e?q)ert systems are 
worth little if the people operating them can not use them properly. Therefore, user training 
should be considered a critical success factor in the successful deployment of software. 
(Holek, 1994) 
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Expecting users to be computer literate is not an assumption that can always be made. 
Some people are computer illiterate, suffer from computer phobia, or are changes in 

the workplace. Also, just knowing how to use a computer does not guarantee that a person 
can operate a spreadsheet package, word processor, or an expert system. Users must be 
trained to use the systems they will use in their line of work. Not only on a basis of what 
button to press, but on a higher level, so that users can do some troubleshooting on their own, 
and are able to at least assess what the problem is and contact the right person for help. User 
training will take time, effort, and money. (Nelson, 1995) 

The questions that need to be addressed are: 

• Who should be trained? 

• How much training should be done? 

• Which methods should be used for training the users? 

• Who should do the training? 

Devising the training plan to answer the above questions correctly will contribute to a 
successful deployment. 

Lastly, the training for a software application will not merely mean teaching a user 
how to input data or how to get answers from the computer. Tr ainin g also means educating 
the user in the overall purpose of the system: what it is supposed to do, how does it do it, 
why do we want or need it done, and who must do it. Effective training does not just tell the 
users how important their contributions are, it lets them see that they are an important, 
integral part in the entire process and that the success of the implementation depends on how 
well they learn what is being taught. (Multinovich and Vlahovich, 1984) 
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2. Training Plan Formulation 

A properly devised training plan will ensure a successful deployment. The software 
should have already been demonstrated to potential users via presentations and 
demonstrations that have informed them about its capabilities and relevance to them. The 
actual trainii^ should provide sufficient, detailed information and hands on experience to use 
the new system effectively (Bouldin, 1989). A framework for building an effective training 
plan includes: (Nelson, 1995) 

• Assessment of training needs 

• Development of training materials and methods 

• Training of trainers 

• Preliminary evaluation of training methods and materials 

• Formal training and learning 

• Evaluation of training 

By paying proper attention to filling in the details of the above steps, training will result in 
users who are able to operate the system effectively and efficiently, who know who to contact 
in the event of a problem, and who are aware of the capabilities and purpose of the system. 
The following methodology developed by Nelson (1995) was used to develop the training 
plan in Appendix B of this thesis. 

a. Assessment of Tridning Needs 

In assessing training needs, it must be determined who to train, what kind of 
skills need to be trained, what kind of methods of trmning to use, and who is to carry out the 
training. On the level of the user, the deployment team must decide what kinds of computer 
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skills each person needs to be able to perform his or her tasks efficiently and productively 
(Prerau, 1990). An assessment of what systems or pieces of software each individual needs 
to be able to use will be done. A dedsion on whether the user needs general use-of-computer 
training or only system specific instruction has to be made. The opinion of each individual 
shoidd be heard with r^ards to interests, actual needs and preferences. On the level of tasks 
to be performed within an organization, an assessment of what are those tasks and who 
should be trained to perform those tasks needs to be made. A good practice is to ensure that 
all relevant personnel become qualified to use the new software. As with any computer 
system, the organization should have its expert users, and each user who needs to use the 
software should possess adequate skills. (Nelson, 1995) 

Within the last three years, personal computers have become increasingly 
available to business and government organizations. Even with the widespread use of 
personal computers, training must address the user who may have no to little experience in 
using personal computers. The training may therefore have to include the basics of installing 
the software and accessing the program from the operating system. (Leinonen, 1995) 

Training needs to be given to everyone associated with the system. This 
requirement includes the following personnel: (Holek, 1994) 

• Upper management; Acquaint upper management with the capabilities and 
limitations of the system. 

• Line management: This training will enable line management to understand how 
the users input data to the system and how they use the outputs. The training will 
allow line management to understand what the software does and how it does it. 
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• Users: This is an opportunity to stress to the users the advantages that the system 
has to offer them. The goals of this portion of training is to provide the users with 
the necessary skills to: 

- Input data into the system. 

- Use the output from the system. 

- Utilizing all the embedded help functions. 

- Keep the system performing the way it was meant to. 

It should be stressed that training should not only teach personnel how to operate the 
software but also teach persoimel about the purposes of the software. 

Based on the different audiences that the deployment team will be training, 
different methods of training will have to be used. Training can be arranged into four 
different categories: (Cousins, 1994) 

• Seminar style presentation: Provide trainees with a basic understanding of the 
operation and purposes of the new software. 

• Classroom/Hands-on: Provide trainees with an introduction to the capabilities and 
purpose of the new software. In addition, users will receive instruction on the 
advanced operations of the software, e.g., data input, following troubleshooting 
paths, using online help, etc. 

• On-The-Job training: The objective of this type of training is to have intermediary 
trainers provide training to users who missed the initial training or entered their 
poation after the software was introduced. This type of training can also be used 
in the event of software upgrades or additions. 

• Follow-up Sessions: The objective of this phase of the training will be to monitor 
the user’s performance with the new software. The duration should be 
approximately once every three months and not last longer than one hour. 

The users and line management should attend classroom presentations to teach them about 

the benefits and purpose of a software application. The users remain at the t raining site to 

receive the hands-on portion of the training. A seminar presentation can be given to the upper 

management in conjunction with a follow-up visit to each department in the organization. 
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These four types of training will ensure that everyone involved with the new software receives 
training that is tailored to their specific needs and interests. (Cousins, 1994) 

b. Development of Training Materials and Methods 

The actual training may consist of a combination of the four different 
categories of training. A training plan should contain the following elements; (Leinonen, 
1995) 

• The purpose and objectives of the training 

• The required training materials 

• A timeline of training events 

• A detailed outline of what will be covered 

• An evaluation form for the trainees to complete on the effectiveness of the training 
The objective of the finished training plan is to ensure that the training is completed effectively 
and accounts for all the requirements of an off-site training location. 

c. Training of Trainers 

It is a false assumption that just because someone knows how to do 
something, then they can just as well teach other people how to do it. There are two possible 
approaches to selecting and training the actual trainers; 

• Train personnel with good communication skills to use the software. 

• Train a user-expert to train others. 

The attitude and approach of the trainer are very important. If the trainer is unable to deliver 
the material to the trainees, the results of the trmning will most probably be ineffective. The 
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use of an outside consultant or even the programmers of the software may be cost-prohibitive 
and may require too much effort in the “train the trainers” area. (Nelson, 1995) 
d. Prdiminmy Evaluation of Training Methods and Materials 
Before starting the actual training, an evaluation of the training plan must be 
performed. The main points to be conadered are whether the training is providing the trainee 
with the desired and needed skills, whether the target group is the right one, and whether the 
methods are effective. The easiest and most effective way to do this is to forward a copy of 
the training plan to the deployment team, software development personnel, and users for 
review. The goal is to let the interested parties have some input into what kind of training 
they will recdve. After the responses are received, necessary changes are made and reviewed 
by the training team before deployment. (Leinonen, 1995) 

& Formal Training and Learning 

The actual training will be conducted according to the training plan, but not 
too rigorously—if new problem areas emerge during the actual training, they should be dealt 
with promptly. Prop^ training of users has already been identified as a critical success factor. 
If not carried out properly, it can lead to costly delays, problems and dissatisfaction on the 
part of the users (Le Compagnon and Leydon, 1991). In general, the tr aining should be a 
balance between knowledge-based training and skills-based training. The training plan should 
have also been geared to the specific groups that need to learn about the new software. With 
these considerations in mind while formulating the training plan, it is anticipated that the 
major training pitfalls of inapplicable training material and ineffective trainers will be avoided. 
(Nelson, 1995) 
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f. Evaluation of Training 

At the end of the training sessions, an evaluation will be distributed to the 
trainees. The questions in the survey should assess the following areas: (Cousins, 1994) 

• Presentation of the material. 

• Skills of the presenter. 

• Whether the trainees felt the objectives of the training were met. 

• Classroom facilities. 

The trainers will assure the trainees that the evaluations will be read carefully with the goal 
to improve the training for future participants. For training to be effective, it must be seen 
as an ongoing process which begins with a determination of institutional and individual needs, 
involves both users and trainers in the planning process, and includes procedures for 
evaluating the effectiveness of the training and making changes and adjustments as needed. 
The training evaluation has been proven to be an effective means of fine-tuning individual 
training programs. It is also a means of determining spedfic areas for which adequate tr aining 
is not being provided. However, a survey will not provide much information on some of the 
questions which the trainers need to consider if training is to be a part of the deployment 
process. (Le Compagnon and Leydon, 1991) Some of these questions on the survey include: 

• Does the training make the user more self-sufScient? 

• Does it make the user more eflScient at his job? 

• Does it provide a more cost-effective means of support to users than other types 
of support? 
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With a well thought-out training plan and a survey that queries the users about the training, 
the training can be conducted efficiently and productively while being improved in the areas 
that trainees specify if needed. (Le Compagnon and Leydon, 1991) 

3. Conclusions About Training 

There is no one right approach to effective user training (Le Compagnon and Leydon, 
1991). Effective training can take on any number of different forms, not all of which take 
place in the traditional classroom environment with an instructor. Documentation, online 
tutorials, and video tapes can provide alternative, adequate, and cost-effective training for 
most user needs. These alternatives, however, can not replace the responsiveness of an 
instructor in person. The training must also be seen as a process to strengthen long-term 
institutional goals and as a cost-effective means of providing user support (Le Compagnon 
and Leydon, 1991). Often, not enough time is spent up front assessing needs and p lanning 
the training effort, nor is enough time spent evaluating the results and making necessary 
changes. With prop©- planning and evaluating, training can be one of the most cost-effective 
means of providing computing support to users. 

D. EFFECTIVE COMMUNICATIONS BETWEEN USERS AND DEVELOPERS/ 

MAINTENANCE TEAM 

A successful implementation does not end with the deployment of the software. Once 
the software is deployed there must be some mechanism for the users to communicate with 
the developers and/or the maintenance team. Lines of communication should be a two-way 
street that provide foUow-up and feedback. The users must have a point of contact in the 
event that they have a question or complaint about the software. Information should also be 
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available on project objectives, status, changes, organizational coordination, and user needs 
(Schultz, Slevin, and Pinto, 1987). Providing a point of contact also provides an 
organizational “home” for the software. The need for assigning a system manager has been 
clearly demonstrated (Keen and Morton, 1978). The system manager that is handling the user 
support should be: (Keen and Morton, 1978) 

• Familiar with the job of the users 

• Skilled technically, administratively, and interpersonally 

• Involved in planning, priority setting, and decision making in the project 

An effective system manager also facilitates the institutionalization of the software while at 
the same time encourages and makes easier user participation (Multinovich and Vlahovich, 
1984). A single point of contact also avoids conflict by having one person prioritize requests 
from the users. 

E. post-implementahon evaluation 

It is impossible to determine implementation success without a thorough evaluation 
(Cerullo, 1979). The determination of success should be based on three factors: (Multinovich 
and Vlahovich, 1984) 

• A prior definition of “improvement”. 

• A means of monitoring progress toward the goals. 

• A review process to determine when the system is fully implemented. 

In the case of a newly implemented system, performance will be based on the accuracy of the 
syst«n resulting from the user’s use of the system. Measures of effectiveness will include the 
realization of any of the benefits projected from the cost-benefits analysis. Satisfaction will 
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be based on the user’s opinion of the software. A survey, an example of which is included 
as an appendix to this thesis, should be distributed to measure ease of use, perceived 
accuracy, and the perceived impact on the performance of the user’s duties (Multinovich and 
Vlahovich, 1984). The survey should be designed to obtain all the infonnation needed to 
judge user satisfaction while remaining as unobtrusive to the user as possible. But how can 
user satisfaction be measured? Lucas and Ginzberg (1990) have formulated a model for 
assessing implementation success as shown in Figure 3-1. 


Acceptance-^Use-►Performance-►Satisfaction 


Figure 3-1. Lucas-Ginzberg model of implementation success. 

The model shows that in order to ensure that users use the new application, they must be 
satisfied with it. A survey can gauge user satisfaction and help determine where areas of 
improvement lie (Lucas and Ginzberg, 1990). 

F. HARDWARE IMPLEMENTATION ISSUES 

While choosing the right hardware is obviously an important choice in a successful 
implementation, there is no standard hardware configurations that are defined for the different 
categories of expert systems due to the unique requirements that every system has. The 
starting point for detamining the hardware that will eventually be deployed should start with 
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an analysis of the minimum hardware required to effectively run the software (McGaha, 
1994). Factors that should be considered prior to deployment are: 

• Portability: Do the users or the environment require a portable computer? 

• Commercial-off-the-shelf vs. Custom design: Is the system able to run on a 
commercially available computer or does it require unique hardware or an 
uncommon configuration? 

• Operating environment: Is the operating environment of the deployed system harsh 
enough to require ruggedized computers? 

• Cost: Is affordability a concern? 

• Expandibility: Does the hardware need to be expandable in anticipation of adding 
new features or upgrading an application? 

• Security: Does the application require a secure operating environment? 
Hardware should be chosen after the development of a full prototype version is completed 
at which time it is easiest to determine the needs of the system (Prerau, 1990). 

G. SUMMARY 

This chapter examined implementation issues that must be taken into account before 
deploying a software application. Obtaining management support, involving users in the 
development of software, training users, establishing effective communications, and 
conducting a post-implementation survey are important issues to implementation success. 
Finally, the type of hardware that the software will be deployed on must take into account the 
six factors listed in Section J. In his thesis, Leonard (1996) lists several factors that are 
critical to implementation success based on published research: (Leonard, 1996) 
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• Create reaKstic expectations about expert system capabilities. 

• Involve the users in the development and implementation plan to the degree that 
they feel they have influence in the process. 

• Promote system usage through organizational rewards. 

• Provide a system of training that instills confidence in the -users and offers the 
opportunity for additional instruction on request. 

• Build enthusiastic support at all levels of management. 

• Establish effective lines of communication between the users and the maintenance 
team by establishing a help desk and points of contact of system advocates. 

The implementation issues examined in this chapter will be applied in Chapter V to the 

implementation of the MK 92 MAES full prototype version. 
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IV. IMPLEMENTATION ISSUES OF MAES AS A SOFTWARE 

SYSTEM 


The previous chapter addressed general implementation issues involved in deploying 
a software application. This chapter will apply the implementation issues'discussed by Prerau 
(1990) spedfically to the initial deployment of MAES. Section A provides an analysis of the 
general software implementation issues that affect MAES. It begins with a description of the 
MAES development history and how it has followed the fi'amework developed by Prerau 
(1990). Section B presents lessons learned by other DOD activities that implemented expert 
systems to diagnose weapons systems. Section C examines organizational structures within 
the Navy and thdr impact on implementation. Section D applies research on management and 
user perspectives to the implementation of MAES and Section E is a summary of the chapter. 
A. MAES DEVELOPMENT HISTORY 

The general software development life cycle model used by the MK 92 MAES 
development team generally follows a model described by Prerau (1990). He divides the 
expert ^stem life cycle into three phases: the initial phase, core development phase, and final 
development and deployment phase (Prerau, 1990). A summary of the MAES development 
for these phases follows. 

1. Initial Phase 

The initial Prerau phase consists of project start-up, selection of the domain, and 
selection of the development environment. In 1991, engineers at NSWC-PHD started the 
devdopment of an expert system designed to diagnose problems associated with the MK 92 
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MOD 2 FCS Daily System Operability Test (DSOT). The domain expert for analyzing and 
documenting the knowledge to be incorporated in the system was a technical representative 
from UNISYS with over 25 years engineering experience. The knowledge for MAES was 
acquired from MK 92 technical manuals, the domain expert’s analysis of the system, and 
heuristics of other MK 92 experienced engineers. 

The development environment chosen for MAES was an expert system shell from 
SoftSell called Adept™ v2.2. This tool was selected after extensive evaluation of diagnostic 
expert system shells and the experience of an Army development team buil ding an ejq)ert 
system for the Ml A1 tank’s turbine engine. Adept was selected for several reasons. First, 
Adept uses a graphical, procedural approach to knowledge representation that is suitable for 
diagnostic systems. Second, it has an integrated graphical user interface (GUI) screen builder. 
In addition, the tool is easy to use and provides for unlimited run-time versions. Finally, the 
development version runs on Windows based 80486 platforms and is relatively inexpensive 
at $695.00 per copy. 

2. Core Development Phase 

The core development phase consists of the development of a feasibility prototype and 
a full prototype system. An initial prototype of MAES was buUt and demonstrated. A full 
prototype, consisting of the calibration and performance modules, was then developed and 
fielded to the USS Sides (FFG-14) and the Fleet Training Center in San Diego (FTC-SD)for 
feedback. The prototype deployment allowed senior technicians both on the ship and at the 
MK 92 technician school to make helpful suggestions. Examples include: 
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• The use of photos to asast in following recommended troubleshooting/repair steps 
and for which pictures are not available in the tech manuals. 

• The inclusion of “why” buttons to explain to a technician why a certain step is 
recommended. This feature also enhances the training capability of MAES. 

• The listing ofpart numbers ^)^ilen a part is recommended for replacement. This will 
speed the supply administrative process of acquiring a new part. 

Bradley and Hauser point out that when technicians observe their suggestions being turned 

into program features it gives them a sense of accomplishment and involvement in a project 

(Bradley and Hauser, 1995). All recommendations were incorporated into the system. 

Involving the instructors at FTC-SD also served the purpose of introducing MAES to student 

technicians that were about to enter the fleet. This was a method to ejqpose potential users 

to MAES prior to reaching thdr ships. Student technidans at the MK 92 “C” school also had 

the advantage of receiving the introductory MAES training over the course of a week instead 

of the one day training schedule planned on for the initial deployment. 

3. Final Development and Deployment Phase 

The MAES project is currently in this phase. During the development and deployment 
phase, a production system is built and the system is deployed. MAES will then transgress 
to the maintenance phase, with software and knowledge bugs being fixed and new info rmati on 
and features being added as necessary. To produce the initial deployment version of MAES, 
several issues were examined. 

• Possible representation and reimplementation: Although a prototype of the 
performance module had been completed, an outside contractor was hired to 
redesign the software, incorporate new knowledge into the module and ensure that 
the module was robust and reliable for fielding. 
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• Deployment Hardware: Factors that were considered for MAES hardware 
dq)loyment were cost, reliability, portability, and screen characteristics. This topic 
is discussed in greater detail in Chapter V. 

• Deployment software: The deployment software is simply a run-time version of the 
development software. Users will be able to run MAES without having to install 
the development software, Softsell’s Adept 2.2™, on their computer. Users will 
not be able to make changes to the distributed software. 

• Input/Output formats: Comments from users have reflected that the user interface 
is extremely easy to learn and use. In recent tests, non-MK 92 technicians were 
able to diagnose MK 92 faults correctly using MAES, even though they had no 
prior exposure or experience with the MK 92 FCS (Myer, 1996). 

• Testing: The initial version of the calibration module has been tested and 
evaluated. FTC-SD has been verifying knowledge by allowing student-technicians 
to use MAES to diagnose faults on a MK 92 mock-up. Knowledge and software 
validation and verification have been part of the development process. 

• Documentation: NSWC PHD will be maintaining MAES after the deployment of 
the initial version. The following documentation to support maintenance is being 
assembled for transfer to NSWC PHD: System Level Description, Software 
Design Document, MAES User’s Manual, Version Description Document, and a 
Program Package. Final versions will be produced as part of the production 
decision to deploy to all ships. 

The implementation plan for the deployment of the initial version provides a guideline for 
completing the steps required for deployment of MAES and is included as Appendix A to this 
thesis. 


The long term maintenance of MAES will be conducted by NSWC-PHD. Having 
NSWC-PHD as the only maintenance source allows a centralized approach to maintenance 
(Prerau, 1990). Work has begun on defining and preparing the deliverables required to 
support the fielded system. 
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B. INCORPORATING LESSONS LEARNED THAT MAY IMPACT 

IMPLEMENTATION 

This section addresses the subadiary research question “What are the lessons learned 
in the implementation of other ejq)ert systems for weapons systems within DOD?” Although 
personal computers and software applications are widely used throughout DOD, expert 
system software is a relatively scarce technology that is still not common within the military. 
However, two large expert systems have recently been deployed by the Army and Navy: 

• The Turbine Engine Diagnostic (TED) expert system is used in the Army 
intermediate maintenance activities to aid gas-turbine technidans in 
troubleshooting the turbine engines installed in the Ml Abrams main battle tank. 

• The Phalanx Integrated Maintenance System (PIMS) used in the Navy to aid fire 
control technicians on surface ships in maintaining and diagnosing faults in the 
Phalanx close-in weapon system (CIWS) was deployed over the past two years. 

Valuable lessons learned from the deployment of these two systems can be utilized to avoid 

problems in the MK 92 MAES deployment. The lessons learned from these systems are 

espedally applicable to this deployment because: 

• Both expert systems were deployed for military systems. 

• Both expert systems have requirements similar to the MK 92 MAES in that they 
were both designed to aid technidans in the maintenance of complex, technical 
machinery. 

• Both expert systems are used primarily by enlisted technicians. 

Video teleconferences (VTC) with the Army Research Laboratory (ARL) at Aberdeen 
Proving Grounds (which developed and deployed TED) and the Naval Ordnance Station 
(NAVORDSTA) at Louisville, KY, (which developed and deployed PIMS) were held. Many 
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valuable lessons were learned in these teleconferences. They fall into several categories 
defined below. 

• Training: Lessons learned that improve training methods and matftriak during 
initial implementation. 

• Maintenance: Lessons learned that improve the post-deployment maintenance of 
the ejq)ert system. 

• Post-implementation evaluation: Lessons learned that improve the conduct and 
effectiveness of the post-implementation survey. 

• Ebrdware issues: Lessons learned that may aid in making choices in which type of 
hardware to deploy MAES. 

The following sections discuss the lessons learned in more detail. 

1. Training Lessons Learned 

The training lessons learned include: 

• Training needs to allow for a wide range of computer experience. Trainers should 
be prepared to deal with computer illiterate personnel. 

• Training teams need to visit each site to conduct the initial training. Besides the 
obvious benefits to the new user, the developers learned much. Trainers were able 
to stand beside the technicians as they observed how the user used the expert 
system and saw how they reacted. As a result they were able to gain first hand 
knowledge on where the deficiencies were in the user interface. 

• Users need to be informed that the issued computer is only to be used for 
accessing the expert system. Maintenance teams received many calls about 
software conflicts resulting fi-om the installation of unauthorized software and 
there is a real threat of the introduction of viruses. 

• Training needs to be coordinated with the commands receiving the expert system 
as far ahead of time as possible. A lack of coordination between activities, will 
likely result in dfficulty getting the right people to attend training sessions. ’ 

• Deployment teams should not install multimedia training aids on computers. A 
training video installed on laptop computers was not found to be worth the time, 
money, and hard drive resources required. 
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2. Maintenance Lessons Learned 


The maintenance lessons learned include: 

• The deployment team should solidt users for suggestions during training sessions. 
Major revisions to the software were made after deployment based on user 
feedback. 

• Developers should be sensitive to the varying levels of computer expertise that 
enlisted technicians may have. Expect to add more on-line help for the 
inexperienced user. 

• A single point of contact should be designated for problems with the software. A 
single point of contact provides consistency to the users that have problems with 
the software. 

• Technical support personnel must be prepared to deal with general computer 
problems. A large number of requests for assistance were for problems 
unassociated with the expert system, such as computers that would not boot or 
problems with the Windows operating ^stem. Plan ahead and prepare for these. 

3. Post-Implementation Evaluation Lessons Learned 

The post-implementation evaluation lessons learned include; 

• The maintenance team should have in place a web page for technical support. 
Extensive use of the Internet was made to make revisions available for download 
and to provide a method for technicians to provide feedback and bug reports. 

• The maintenance team should include a survey that can be answered by individual 
users on a web page. It was found that technicians were not answering surveys 
that were mailed out. Once world wide web access was available, the problem of 
collecting data diminished. 

4. Hardware Lessons Learned 

The hardware lessons learned include: 

• All technicians should have access to the computer that contains the expert system. 
Some senior technicians kept the computers locked up for fear of pilferage, 
limiting access to others. 
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• Training should emphasize that this system is for troubleshooting purposes only. 
Some senior technicians used the computers for other purposes, such as 
administrative tasks. 

• Ruggedized laptops are not necessary. Offtheshelflaptops have so far fared well 
in an intermediate maintenance garage type environment. 

• The computers that are distributed should have durable pointing devices. 
Trackball and mouse-type pointing devices tend to become inoperative in a 
maintenance environment. 

• At a minimum, dual scan screen technology should be used for laptop computers. 
If the laptop will be used in bright sunlight, then an active matrix screen should be 
specified. Technicians had difficulty discerning details in photos when black and 
white displays were used. 

C. ORGANIZATIONAL STRUCTURES AND THEIR IMPACT ON 
IMPLEMENTATION 

Lucas and Ginzberg (1990) developed an implementation model that addresses 
management and user concerns in the implementation process Before the factors in the sub¬ 
models can be examined, the organizational structure of the ship and the shore-based 
maintenance and training fedMes must be considered. A determination of who fits the roles 
of management and user must first be resolved. 
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1. Shipboard Organizational Structure 

An organizational chart of the typical chain of command onboard a FFG-7 is shown 
below in Figure 4-1. 



Figure 4-1. FFG-7 Chain of Command. 


System operation and maintenance aboard ship are conducted by enlisted technicians who 
possess the Fire Control (FC) rating with a Navy Enlisted Classification (NEC) of 1102. The 
technidans receive this NEC once they have completed the 32 week MK 92 “C” school. The 
manning level for MK 92 FCs on FFG-7 class ships is seven, but may vary fi'om ship to ship. 
Reserve ships are allotted one less junior technician. Budget cuts and downsizing have 
contributed to the lack of proper manning levels in the fleet. Reduced manning is a 
contributing factor for the high number of technical assistance requests. (McGaha, 1994) 
Holek (1994) describes three types of personnel that need to be trained: upper 
management, line management, and users. Applying this framework to a shipboard 
organization would result in the following classifications: 
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• Upper management: The commanding officer, executive officer, and combat 
systems officer. 

• Line management: The division officer and division chief. 

• Users: All MK 92 technicians; junior and senior. 

Classifying shipboard personnel in the above manner will allow the MAES deployment team 
to tailor their training plan to each category of manager or user. 

2. Shore-based Maintenance and Training Organizations 

Unlike the shipboard organization, there is no hierarchy associated with the shore- 
based maintenance and training commands. After enlisted recruits graduate from boot camp 
and “A” school, they are sent to the “C” school for which they have qualified. In the case of 
MK 92 technicians, the “C” school is located at FTC-SD and FCTC Dam Neck. Although 
the schools are not a part of the maintenance organization, they have proved invaluable as a 
source for testing the knowledge contained within MAES. The school has also served as a 
way to expose new MK 92 technidans to MAES in a training environment and to gather da t a 
on the system. 

Technical support is available to shipboard technicians from Fleet Technical Support 
Centers (FTSCs) and NSWC-PHD. A ship requests technical assistance through the 
respective type commander’s (TYCOM) Readiness Support Group (RSG) via telephone or 
CASREP. FTSC’s are staffed by both senior Navy enlisted technicians and experienced 
civilian technicians. FTSC technical representatives can be considered line mflnagfimftnt due 
to their position as technical experts. However, there are different levels of experience and 


52 




expertise at the FTSC. Some technidans may, on occasion, use MAES in areas they are less 
knowledgeable about. They would be considered users in such cases. 

D. MANAGEMENT AND USER PERSPECTIVES ON MAES 

IMPLEMENTATION 

The Lucas-Ghizberg (1990) model of implementation consists of two stages: adoption 
of a system by the managers and adoption of the ^stem by users. This model is applicable 
to the FFG-7 shipboard organization. The following sections discuss these two areas. 

1. Management’s Perspective of MAES Implementation 

The importance of management support and its contribution to implementation 
success was discussed in detail in Chapter IH. Although the two sub-models from the Lucas- 
Ginzberg (1990) implementation model include many factors that influence implementation 
success, only the specific factors that can be influenced by the MAES development and 
deployment teams are addressed here. 

• Manager acceptance: On a US Navy surface combatant, enlisted tec hnician s are 
influenced in their work habits most strongly by the division chief and somewhat 
by the division ofiScer. Manager acceptance will in turn be influenced by manager 
knowledge of MAES and manager assessment of the MAES system and available 
support. 

• Manager knowledge of MAES: The MAES deployment team will instruct division 
officers and chiefe on the benefits and purpose of MAES and the basics of how the 
system works. Acceptance of MAES should increase with management education. 

• Manager assessment of MAES and support: Showing officers and chiefr the 
advantages and benefits that MAES equipped tec hnicians will have over 
technicians not using MAES should gain their support for the system. Provi ding 
the division officer and chief with a means of repairing hardware faults and 
providing technical support for MAES software will gain their confidence of the 
system. 
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• Manager belief in the MAES concept: OflScers and chiefs who believe in MAES ’ 
ability to aid technidans, cut down the NFE problem, and reduce MTTR will want 
to learn about the ^stem. With this knowledge they will encourage the technicians 
to use it. 

• Top management support; The deployment team should address the commanding 
officer, executive officer, and the combat systems officer during an informal, half- 
hour brief on MAES. This would provide an opportunity to inform the ship’s 
upper management of MAES benefits. A short demo should be included. This is 
an important brief as the ship’s upper management, if they believe in the system’s 
importance, can greatly influence how the division officer and chief view MAES. 

Obtaining upper management’s support can significantly increase the probability of a 

successful implementation of the system (Lucas-Ginzberg, 1990). 

2. User’s Perspective of MAES Implementation 

Improving the technidan’s ability to correctly diagnose and solve faults in the MK 92 
MOD 2 PCS is one of the primary goals of MAES. A short discussion of each of the 
applicable user variables in the Lucas-Ginzberg model that can be influenced by the 
implementation and how they may be applied to MAES follows. 

• User acceptance; This is a key variable on whether or not a system will be used. 
It is influenced by the user variables discussed below. The instruction, demos, and 
hands-on training should all be oriented at achieving user acceptance. 

• User knowledge of MAES: For users to accept a system, they must be 
knowledgeable of it. The MAES deployment team will train the technicians on 
how MAES functions, its capabilities, and its limitations. 

• User assessment of MAES and support: Training of the users must ensure that 
they have the necessary means to resolve hardware and software problems. A 
single point of contact for providing MAES support should be established. 
NSWC-PHD, as system configuration manager, will be so designated. 

• MAES characteristics: This variable represents the features and capabilities of 
MAES. Easy to use systems such as MAES are more likely to be accepted by the 
user. The implementation process should cover this variable. 
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• User-researcher involvement: Although it was not possible for every technician to 
be involved in the MAES development, users should be made aware of the fact 
that some of the best features of MAES have resulted from sailor’s suggestions 
during shipboard prototype demonstrations and evaluations at the training centers. 
This should provide a sense of user ownership of the system. 

• Us©" perception of management support: The division officer and chief must foster 
an environment of support through their encouragement of MAES use when 
diagnosing applicable MK 92 casualties. Users should also be aware of upper 
management’s support for MAES. Encouraging MAES use as a training tool to 
study for rating exams and as a tool for conducting training sessions will aid user 
perception of management support. 

• User knowledge of system purpose and use: Training in this area will be 
concurrent with the variable “user knowledge of MAES”. The difference is the 
focus on the purpose of MAES. Technicians need to be made aware of the high 
cost of NFEs, the high MTTR, and the increasing reduction in tech assist support. 

• Problem urgency: This variable reflects the criticalness and urgency of the 
problems (NFE’s, high MTTR, reduction in tech support) to which MAES 
addresses. Educating the technicians on the benefits of saving them time and 
saving their ship money increases their stake in MAES’ success. 

• Organizational support: This is a measure of the degree to which the deployment 
team can convince management to allow technicians the time to train on MAES. 

Most of the above variables can be positively influenced by how effectively the MAES 

deployment team conducts the training during their deployment visits to the FFGs. A well 

thought-out and organized training plan that takes these variables into consideration will be 

a crucial factor in the implementation’s success. 

E. SUMMARY 

Although MAES is an expert system, general software implementation issues should 
not be ignored. Adhering to an established and tested development methodology, such as 
Prerau’s, provides a foundation for a successful implementation. Also, the experience that 
other organizations have in implementing a similar system can be invaluable. If problem areas 
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were encountered in prior e^ert ^stem implementations, they can be avoided by determining 
what lessons have been learned. 

To properly address each level of an organization that may use MAES, the 
organizational structure and how it impacts an implementation must be understood. T hen , 
an implementation may be developed that addresses the concerns of both the managers and 
the users. The next chapter addresses implementation factors that are specific to the MAES 
deployment. 
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V. IMPLEMENTATION ISSUES FOR THE INITIAL 
DEPLOYMENT OF MAES AS AN EXPERT SYSTEM 


This chapter applies the implementation methodology and issues discussed in the first 
four chapters of this thesis to the implementation of the initial version of the MK 92 MAES. 
Section A classifies MAES in accordance with the Meyer and Curley (1991) scheme and 
applies expert system implementation fectors specifically to MAES. Sections B and C discuss 
obtaining chain of command support for MAES and involving MK 92 technicians in the 
implementation, respectively. Sections D and E recommend approaches to training and 
supporting MAES users. Section F explains how to conduct and evaluate a post¬ 
implementation survey while Section G addresses hardware implementation issues. Section 
H is a summary of the chapter. 

A. EXPERT SYSTEM IMPLEMENTATION FACTORS 

As previously discussed, ^ert systems possess unique characteristics that distinguish 
them fi-om traditional management information systems. The fact that expert systems also 
vary from one another also needs to be taken into consideration. The following sections 
discuss how MAES is classified as an expert system and additional considerations that need 
to be made for implementing MAES. 

1. OassifyingMAES 

Meyer and Curley (1991) developed a classification scheme that views expert systems 
along two attributes: knowledge complexity and technology complexity. MAES best fits into 
the cat^ory “Knowledge Intensive”, since it incorporates the knowledge of skilled decision 
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mako's, requires a simple computing environment, and aids the decision making for a specific 
group. MAES has the following characteristics: 

• MAES incorporates the knowledge of skilled and ejqperienced engineers. 

• MAES requires a simple computing environment. MAES is a point-and-click 
^stem that runs efBciently on a 486-DX4 75 MHz laptop with 8 Mb of RAM. 

• MAES aids decision making for a specific group. This group includes enlisted 
technicians and technical representatives responsible for maintaining the MK 92 
MOD2FCS. 

• MAES was developed using an expert system shell. 

• MAES contains complex knowledge and represents true e?q)ertise, instead of 
commonly used procedures. 

These characteristics therefore most closely are associated with the “ICnowledge Intensive” 
category. Knowing which category of expert system that MAES falls under aids in 
implementation planning since each type of expert ^stem is affected differently by 
implementation factors (Bradley and Hauser, 1995). 

2. Expert System Factors That Affect MAES Implementation 

The feet that MAES is an expert system carries with it additional considerations for 
implementation. These were addressed in general in Chapter HI. Figure 5-1 is a summary 
of how Bradley and Hauser (1995) rate the relative importance of implementation factors for 
different e3q)ert systems. MAES’s category is encircled. Knowing which factors will be most 
influential in the implementation and applying them increases the probability of a successful 
implementation (Bradley and Hauser, 1995). Also, since MAES is a resource constrained 
project (both in funding and persoimel), every fector may not be able to be fuUy addressed. 
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Determining where efforts should be concentrated during the implementation process will 
reduce the risk of failure. 


Expert System Classification 


Factor 

T 

11 

HI 

IV 

Innovation Characteristics 





User perception of computing technoIog\’ 

H 

H 

H 

H 

User perception of expert systems 

L 

M 

M 

H 

Organizational influences 





User involvement in expert system design 

H 

H 

H 

H 

User involvement in expert system implementation 

H 

H 

H 

H 

Reward structure 

L 

M 

M 

H 

Training 

L 

M 

M 

H 

Top managemenl support 

L 

M 

M 

H 

Supervisor support 

H 

a 

U 

H 

Advocates 

T. 

M 

M 

H 

Consulting aids 

L 

M 

M 

M 

Individual Differences 





User perception of change 

H 

H 

H 

H 


L ^ low emphasis in the implementation plan 

medium emphasis in the implementation plan 
H = high emphasis in the implementation plan 

Figure 5-1. Relative Importance of Expert System Implementation 
Factors with MAES Category Highlighted. (Bradley and Hauser, 1995) 


From Figure 5-1, the factors that are most likely to have a major influence in the 
implementation of MAES are as follows: 

• User perception of computing technology: Greatly dependent on the complexity 
of the expert ^stem. The uncomplicated graphical interface that MAES uses eases 
the risk of technicians becoming overwhelmed by MAES. A thorough tutorial on 
how to use MAES will be an integral part of the training session and user’s 
manual. All users should receive the training as there is always the possibility of 
a technidan being computer adverse because of a lack of familiarity and training. 

• User involvement in MAES design and implementation: Discussed in the previous 
section, users have provided valuable input on the features of MAES. This fact 
needs to be emphasized during implementation. 

• Supervisor support: Analogous to management support, discussed previously. All 
levels of management need to participate in the implementation. 
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• User perception of change: This factor is associated with the technicians 
perception of how MAES wiD impact their work routine. Emphasizing that MAES 
is just another piece of test equipment (albeit a much more effective one) should 
minimize technician perception of change. 

During the MAES implementation, emphasis should be placed on the technician’s perception 
of the technology, how easy it is to use, involvement of the users in the implementation, and 
gaining management support. 

B. OBTAINING CHAIN OF COMMAND SUPPORT FOR MAES 

As discussed in Chapter HI, management support will be a requirement for the 
successful deployment of MAES. Applying methods of gaining the support of the chain of 
command and overcoming resistance to MAES are discussed in this section. 

Gainmg the support of the chain of command on the ship and of senior technical 
representatives ashore should be the key objective for the MAES deployment team. The 
introduction of a new technology that introduces change into the way maintenance is 
performed will meet with some d^ee of resistance. As developers, one of our key concerns 
is that MAES be used when appropriate. If it is not used, a fair evaluation of MAES can 
never be made. There are several things that may be done. 

1. Selling MAES to the Chain of Command 

Gaining support of the chain of command is the primary reason that the deployment 
team will be visiting each ship the day after training the MK 92 technicians. A short brief to 
the commanding ofiBcer, executive officer, and combat ^sterns officer will inform them of the 
benefits MAES will provide. The benefits that are most likely to appeal to them, and should 
be stressed during the brie^ are MAES’ abffity to save the ship money (through the reduction 
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of NFEs), increased operational readiness, and improvement in the sailor’s quality of life 
through a reduction in the long hours spent troubleshooting faults. The senior chain of 
command must be made aware and convinced that their support will be instrumental in 
convindng the technicians to use MAES. Their position, in the form of a directive which is 
either oral or written, that MAES shall be used for all applicable faults can have a tremendous 
influence. Occasional verbal inquiries on the system, demonstrating their continued interest, 
can also serve as a strong motivator. 

2. Getting Line Management Involved 

The division officer and chiei^ the next two players in the MK 92 chain of command, 
are also instrumental to a successful implementation. They have daily contact that is a direct 
influence over the MK 92 technicians. The division officer and chief may show support by: 

• Attending meetings with the technicians about MAES. 

• Ensuring MAES training is part of the technician’s training plan and providing time 
for them to train on MAES. General Military Training (GMT) is usually 
conducted at least three times a week onboard a ship. Providing time for 
technicians to train on MAES, as little as once a month, will show them that the 
chain of command is serious about the importance of MAES. 

• Becoming knowledgeable about the basic workings of MAES. It is a practice of 
good leadership for officers to know the basics of the equipment for which they are 
responsible. Being familiar with MAES will also allow the division officer to 
better complete the deployment evaluation that is critical to fleet wide deployment. 

• Personally using MAES. If the division officer or chief use MAES, even if just 
once or twice, they will become more familiar with its capabilities and limitations. 
This understanding of the system should provide them knowledge to support their 
insistence that MAES be used. 

Although the demands on an officer or chiefs time are many, it should only take about 15 
minutes to learn how to use MAES (Myer, 1996). If the chain of command is able to devote 
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just a fraction of their time to demonstrate their support for MAES, they play a key role in 
seeing that MAES is utilized. 

3. Generating Support Through Demonstration 

Conducting a brief demonstration of the system to the chain of command would be 
an effective way to present the capabilities and advantages of MAES (Bouldin, 1989 ). The 
demonstration must be simple enough to be brief and comprehensible. It should focus on the 
key features of the system and MAES’ ease of use. Illustrating one example of a MAES 
troubleshooting path and demonstrating its overall capabilities should be suflBcient. The 
MAES demonstration team should keep it simple, brief, and familiar (Bouldin, 1989 ). 

4. Gaining Support Through Respected Advocates 

Gaining the support of MK 92 technical representatives that are respected in their field 
and trusted by the chain of command, will aid in gaining support for MAES (Bouldin, 1989). 
One strategy would be to have a respected MAES advocate available at the MAES 
demonstration onboard the FFGs. The key activities that could provide the respected 
advocates include the Fleet Technical Support Centers, or the ISEA at NSWC-PHD. The 
training centers may also be a source. An advocate with a lot of experience and that is known 
to technicians onboard the ship can provide tremendous support for MAES that will be 
passed on. 

5. Distributing a MAES Newsletter 

Distributing a MAES newsletter to each ship involved in the implementation would 
keep management and technicians involved in the project’s progress and provide a forum for 
discussion. The newsletter would also provide a medium to publicize the contributions that 
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fleet technicians have about MAES and provide suggestions on how to use MAES effectively. 
A newsletter could be published as infrequently as twice a year and be small enough to fill 
only a few pages and still serve as a tool for developing and retaining an ongoing support by 
the chain of command. The challenge will be finding the resources, both funding and 
personnel, to produce such a newsletter. 

6. Presenting Information from the MAES Cost-Benefit Analysis 

Multinovich and Phatak (1981) state that one of the best ways to obtain management 
support for a new technology implementation is by presenting them the results of a favorable 
cost-benefit analy^. Cost-benefit analysis results may provide a means that is easier for the 
senior chain of command to imderstand. Fortunately, a cost-benefit analysis for MAES was 
completed in September 1993 by LCDR Steven Powell, a graduate student at NPS. Key 
results that should be presented at the shipboard briefing are: (Powell, 1993) 

• Approximately 22% of MK 92 parts turned in for repair were No Fault Evident 
(NFE) parts and CASREPs from ships requesting technical assistance required an 
average of 251 hours until repair. 

• The NFE problem is costing the fleet $900,000 in OPTAR funding per year for 28 
Mod 2 ships. Note: the original cost benefit analysis was based on 51 FFG-7s. 
These figures represent current CNO plans of 28 MK 92 MOD 2 deployed 
systems. 

• With the Calibration and Performance modules, MAES can improve operational 
readiness 8% and the mean time to repair should improve by 15%. 

• The two modules in veraon 1.0 of MAES can cover approximately 40% of system 
faults. 
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Since the figures fi-om the cost-benefit analysis are extremely positive, they should be used 
to gamer management support. In addition, the results of recent system tests at FTC-SD 
strongly support the cost-benefit analysis findings. 

C INVOLVmGMK 92 TECHMCIANS IN THE IMPLEMENTATION 

In Chapter HI, research had shown that users should actively be involved in the 
implementation of an expert ^stem throughout its life cycle. Enlisting the aid of technicians 
in the development of MAES has been pmdent since technicians often provide a different 
perspective of a system. More can be learned through their participation than through 
interviews or surveys alone. Multinovich and Vlahovich point out that users have often seen 
solutions that did not work and may have observed solutions that did (Multinovich and 
Vlahovich, 1984). Without their involvement, valuable lessons may be lost. The MAES 
development team has actively sought out the participation of MK 92 technicians. It also 
needs to do so during the implementation. 

1. Past Participation of MK 92 Technicians 

Throughout the development of MAES, prototypes have been given to technicians 
for evaluation and suggestions. The MAES program was provided to the USS Sides (FFG- 
14) for two years and the USS Wadsworth (FFG-9) for six months. They have provided 
valuable ideas and suggestions for the system. 

Instructors at FTC-SD and students have also been included in the development of 
MAES. They have played an extremely valuable role. Their suggestions have resulted in the 
addition of more robust on-line help features; the inclusion of photos in the help sections; 
“Why” buttons that provide insight into the process the domain expert is using to 
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troubleshooting a fault; and the inclusion of supply information that speeds up the ordering 
process. Knowledge by users that thdr suggestions are implemented in the design of an MIS 
stimulates thdr enthusiasm for the system (Bradley and Hauser, 1995). It has been previously 
documented that making such users feel they are important to the design and evaluation 
process by elidting their feedback can contribute to a successful implementation (Lucas- 
Ginzberg, 1990). 

2. Livolving MK 92 Technicians in the Deployment and Evaluation of MAES 

The dose relationship between MAES developers and the te chnician.*; in the fleet and 
the instructors at the training schools is one reason why MAES has so far been well received. 
It is important to continue this cooperation during the maiirtenance of the system. 
Approaches to maintaining some form of communication between the developers and the 
users will be discussed in more detail later in this chapter. Alternatives to obtain input from 
the users include the following: 

• Require the technicians that have MAES to periodically submit a Digital Systems 
Feedback Report (DSFR). This easy to fill out form provides a convenient way 
for technicians to provide suggestions to MAES developers. The form can be 
either frxed or mailed. A copy is included in the MAES User’s manual (included 
as Appendix C). 

• Solidtuso" input with periodic surveys. A sample MAES user survey is included 
as Appendix D. It is discussed in more detail later in this chapter. 

• Solidt responses from each ship via standard navy radio message. While intrusive, 
this method would ensure compliance due to its high visibility. 

• Conduct phone surveys of ships that are using MAES. The disadvantage of this 
method is that it would be man-power intensive and would depend on the 
availability of technicians and whether or not ships are inport. 
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The most useful and cost-effective method would be to distribute a survey to MAES 
equipped ships periodically. A survey would also provide some initial empirical data that 
could be further analyzed. The aspects that may be assessed would only be limited by the 
questions included in the survey. Continued user involvement is important to the successful 
evaluation of the system. 

D. TRAINING MAES USERS 

The focus in this section is training for the initial implementation of MAES. The 
generic questions that were raised in the previous chapter about training are addressed for 
MAES below: 

• Who should be trained? All MK92 fire control technicians and FTSC technical 
representatives. This includes those temporarily assigned other duties (such as 
mess duty). 

• How much training should be done? One day of classroom training with hands-on 
familiarization would be suffident. From feedback received from FTC-SD, if they 
were familiar with basic electronic test procedures and equipment, junior 
technidans were able to use MAES within one hour (Myers, 1996). 

• What training methodologies should be used? Both classroom lecture and hands- 
on training. These two methods provide the users of MAES with the background 
and experience to effectively use MAES. 

• Who should present the training? A combination of instructors is recommended. 
For the deployment of the initial implementation, NSWC-PHD should coordinate 
the training. NFS faculty, &mihar with the history and software development 
should partidpate. The benefit of having a senior sailor, perhaps from one of the 
training centers or FTSCs, would provide a tremendous sense of credibility for the 
system. In addition, FTSCLANT personnel should participate. 

The answers to the above questions provide the framework for developing the t raining plan 

that will be used by the MAES deployment team. 
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The actual training provided MK 92 technidans will need to cover sufScient, detailed 
information and hands on experience in the use of MAES (Bouldin, 1989). Following the 
steps discussed in the previous chapter on formulating a training plan provides an excellent 
outline for proceeding. The following six sections discuss each step as applied to MAES 
training. 


1. Assessing Training Needs 

To assess training needs one must determine who needs to be trained and to what 
level each group should be trained. Earlier in this chapter the shipboard organizational 
structure on a FFG-7 class ship was defined. Tailored training needs to be provided for the 
various organizational levels. FTSC technical representatives, as well as training center 
instructors, also have to be trained. A recommended level of training for the different levels 
in the organization would include: 

• Upper management: The commanding officer, executive officer, and combat 
systems officer should receive a presentation on the capabilities and benefits of 
MAES, as well as a short demonstration of the system. Approximately one hour 
should be allotted. 

• Line management: The division officer and chief should attend the morning half of 
the technician training. They should receive information on the backgroimd, 
benefits, and the basics of operating MAES. 

• Users: The MK 92 technidans and FTSC technical representatives should receive 
the training specified for line management. In addition, a hands on tr aining session 
that covers all aspects of MAES operation should be taught. For the tr aining in 
the Norfolk area, the FCTC facilities should be considered, as actual faults could 
be induced as part of the training. 

The goal of this training is to ensure that each organizational level understands the purpose, 
benefits, and limitations of MAES and that the users are comfortable in its operation. The 
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users and line management should receive their training in a classroom away from the ship. 
Upper management will receive their training aboard their ship. These different levels of 
training ensure that everyone receives training tailored to their specific needs. 

2. Development of Training Material 

The NFS has taken the lead on developing the preliminary materials. The main 
components of the MAES user training program are a training plan and the MAES User’s 
Manual. A preliminary training plan is included as Appendix B and the updated user’s manual 
is included as Appendix C. These documents will be received by NSWC-PHD and will be 
evaluated after the initial implementation on COMNAVSURFLANT ships. The training plan 
includes information about: 

• The purposes and objectives of the training. 

• The required training materials that the deployment team will need to provide to 
cany out their training. 

• A chronological outline of what topics will be covered. 

• A training evaluation form to pass out to all trainees to determine if the goals of 
the training are being accomplished. 

The user’s manual provides information to users who missed the training or need to refresh 
their knowledge of how to use MAES. The MAES user’s manual should contain the 
following information: (Lester, 1996) 

• An explanation of the purpose of MAES and what information the user’s manual 
provides. 

• Step by step instructions on how to install MAES and what the hardware and 
software requirements are. 

• How to initialize, operate, and exit from MAES. 
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Completing the above documentation is the first step in the training process. The next steps 
must be complete before the deployment team can progress. 

3. Training Team Composition 

The training team composition should minimally consist of two persons. One trainer 
should be fi’om NSWC-PHD, the project manager for MAES and eventual system 
configuration manager. The second member should be an experienced Navy technician for 
the MK 92 MOD 2 PCS. Possible sources are a FCC from FCTC Dam Neck or an FCC fi’om 
FTSCLANT. An FCC should be able to relate well with his fellow sailors. The knowledge 
and job of the FCC will also provide credibility to the training team. Because this initial 
deployment is for the evaluation of MAES, a feculty member fi'om NPS should also be part 
of the team. Data will need to be gathered on MAES and NPS will coordinate this effort. 
The focus for the NPS representatives will first be to provide information on the problems 
facing the MK 92 MOD 2 technicians and the benefits MAES offers based on the NPS cost- 
benefit analysis. The second purpose will be to implement the evaluation process. 

4. Preliminary Training Review 

While this research has produced the initial training components, review by others 
knowledgeable of the MK92 is warranted. Therefore, copies of the tr aining plan and user’s 
manual should be forwarded to FTC San Diego, FCTC Dam Neck, FTSCLANT, and MAES 
developers at NSWC-PHD for review. Feedback should be received and acted upon prior 
to conducting the actual training. 
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5. Evaluation of Training 

At the end of the training sessions, an evaluation should be distributed to the attendees 
for their view of training. An example evaluation is included at the end of the training plan 
in Appendix B. Completed evaluations should be reviewed carefully by the trainer at the 
conclusion of each training session. Needed changes should be made. A consensus of the 
training team members should decide on needed changes. 

6. Alternatives to On-Site Training 

Providing MAES users with on-she training is the preferred method of training. It has 
the benefit of addressing questions on the spot, providing dedicated attention to a technidan 
having trouble comprehending some aspects of MAES, and a chance to provide a personal 
brief to the chain of command. Should availability of ships or funding constraints pose a 
problem with providing on-ate training to all ship’s crew there are less expensive options that 
could be considered. Options include; 

• Training ship personnel via video-teleconferencing. This method would save the 
MAES deployment team travel costs. However, it would require considerable 
coordination and may be subject to time restrictions that are assodated with using 
military video-teleconferencing equipment. 

• A videotape could be produced by the MAES deployment team and sent to each 
ship along with the MAES software and hardware. This option has the advantage 
of providing MAES users with a well choreographed and structured training 
sessioa It would not allow the deployment team to address specific questions nor 
address the chain of command personally. 

• Sending each ship printed training materials along with the MAES software and 
hardware. This is the least preferred method, but provides an alternative if funding 
becomes a problem. 
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• Provide training to personnel at one central training facility on the East Coast. 
While this poses the least costly alternative for the training team (in terms of both 
time and money), it puts a heavy burden on the fleet sailors. Not all MK 92 MOD 
2 technidans would likely be able to attend because of the costs (in both time and 
money). The system’s goal is supposed to ease the load on the sailor. This shifts 
the burden fi-om the implementation team to the fleet and is considered the least 
desirable alternative. 

Actual training alternatives may be a combination of the above methods. But the preferred 
method would be the on-site training of MAES users in their homeports by a professional 
deployment team. 

E. SUPPORTING MAES USERS 

This section describes support that should be available to the MAES users after the 
system has been deployed. The MAES configuration manager will be a project engineer at 
NSWC-PHD (Code 4W32). The configuration manager will provide MAES users with a 
single point of contact that is knowledgeable in the technical aspects of the system, both 
hardware and software. 

1. Hardware Support 

If a MAES computer suffers a defect that is covered under the manufacturer’s 
warranty, then the ship will get the computer repaired through the applicable company’s 
warranty program. A computer that sustains damage that is not covered by warranty, should 
have to be processed for replacement. The configuration manager at NSWC-PHD should 
maintain a list of acceptable models and manufacturers available on GSA schedule or other 
sources that meet MAES’ minimum requirements. 
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2. MAES Software Support 

Software support for the MAES program falls into two categories: pre-bundled 
commercial-oflf-the-shelf software and MAES developed software. Pre-bundled software 
includes all software that comes pre-installed on the computers such as the Windows™ 
operating system. MAES users will use the provided documentation and commercial support 
provided by the applicable software company for pre-bundled software. Support for MAES 
software will be provided by the MAES configuration manager at NSWC-PHD. The user’s 
manual provides MAES users with directions for contacting the MAES configuration 
manager. It also provides reporting procedures for software problems and suggestions. 
When upgrades are needed they will be made available by FTP on the NSWC-PHD web page, 
by transmission via the Streamlined Automated Logistics Transmission System (SALTS), or 
by distributing upgrades on diskettes via mail to each ship. If resources are available, 
information on MAES upgrades and user contributions may be available through a MAES 
newsletter mailed to each ship or by publications listed on the NSWC-PHD web server. 

F. CONDUCTING A USER SURVEY 

A user survey will be conducted to determine the value of MAES during the 
evaluation period. It will be used to detemiine user satisfaction, suggestions that MAES users 
have, and for documenting recommended improvements that need to be made to either the 
MAES knowledge or user interface. The MAES user survey should be conducted by mail 
since this is the most cost effective method. It also ensures that aU ships can be contacted 
whether inport or underway. In a mail surv^, the MAES users will have more time to collect 
facts, talk with each other, or consider replies at length than is possible with a telephone or 
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personal interview. Another alternative would be to use e-mail for responses. A proposed 
MAES user survey is included as Appendix D. 

1. Implementing the User Survey 

A primary consideration used in the design of the MAES survey was that the 
respondent should be able to answer the questionnaire in a short period of time, e.g. ten 
minutes (Cooper and Emory, 1995). The following procedures should be used to ensure that 
the survey is returned by as many respondents as possible: (Cooper and Emory, 1995) 

• Follow-ups, or reminders, should be conducted after the survey is distributed in 
order to increase the response rate. 

• Advance notification of the survey should be given by the deployment team 
notifying the MAES users that they will be asked to complete a survey that is 
important for system improvement and will have a significant impact on the fleet 
wide deployment of MAES. 

• The user survey should be designed to be completed by the user within ten 
minutes. 

• Pre-addressed, stamped envelopes should be provided with the surveys. 

• A personalized cover letter should be included with the survey telling the users that 
their opinions are important to the improvement process and for justification for 
fielding MAES to all FCS MK 92 MOD 2 ships. 

• Users should be informed that their anonymity is assured. 

The survey should be conducted at least twice during the implementation of the initial 
evaluation. This will allow the analysts to gauge any changes in user opinion or use du ring 
the initial implementation period. The piimaiy research question for the survey is “What is 
your level ofsatisfection with the MAES software?” A subsidiary research question is “What 
improvements can be made on MAES?” 
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2. Analyzing the Results of the Survey 

The majority of the questions on the survey are answered by the respondent circling 
a number. This type of design decreases the time required to complete the survey and malfftg 
the analysis of the data easier. The target audience will be all MAES users that have actually 
used MAES to diagnose faults in the MK 92 PCS. 

Methods of analyzing the data should include running a frequency of variables in a 
statistical analysis program, such as SPSS. A frequenqr of variables analysis would break out 
which aspects of MAES users like or dislike the most. A histogram plot should be produced 
to determine how respondents rate MAES. This analysis should also be useful in determining 
areas where the software is not meeting the full expectations of the users. Another analysis 
that could be done is an analysis of variances to determine the correlation between how often 
MAES is used with satisfaction. The analysis of variance test could also show if there is one 
aspect of MAES that is poorly designed and deters the user from using MAES. A comment 
area is included to allow MAES users to elaborate on any of the questions they were asked. 
The analysts can categorize comments into categories such as ease of use, suggestions, or 
hardware likes/dislikes to determine if any suggestions are made often enough to warrant 
attention. 

G. HARDWARE IMPLEMENTATION ISSUES 

Part of the evaluation of MAES will deal with an assessment of the computer 
hardware requirements for MAES. During the evaluation, MAES will be deployed on sk 
COMNAVSURFLANT ships using COTS laptop computers. One of the first issues will be 
to determine if laptop computers are suitable for the effective employment of MAES. 
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Another element in the evaluation will be the reliability of COTS computers. Other features, 
such as screen visibility, user input ergonomics, battery life, etc., will also be evaluated. This 
section discusses the subsidiary research question “ What are the hardware implementation 
issues assodated with deploying MAES?” This subject was previously examined by McGaha 
(1994). The section draws extensively on McGaha’s research. It provides updates to his 
recommendations in areas that have changed. 

1. PortabOity 

MAES provides a two-way communication with the technician to diagnose the 
system. It has been designed with features that will enable the technician to expeditiously 
diagnose and repair a fault. These include illustrations on how to carry out a called for 
procedure, pictures, and part number information. 

Since the different components of the MK 92 PCS are located in six different 
compartments of the ship, a laptop computer will provide the portability needed to use MAES 
effectively and is strongly recommended. McGaha provided additional guidance on the use 
of dedicated laptop computers that included: (McGaha, 1994) 

• The computa^ should be classified as a piece of test equipment. This will ensure 
minimal use of the computer for administrative purposes and give first priority for 
its use to maintenance. 

• A laptop computer can be stored more easily and in a secure container. 

• A dedicated hardware platform will minimize the potential for viruses and the 
software conflicts that can occur with other installed software. 

• It ensures that a technician has access to MAES at any time, not only to use for 
troubleshooting, but also for training. 
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2. Commercial-OfT-The-Shelf (COTS) Versus Ruggedized Computers 

Although raggedized computers are designed to withstand an industrial environment, 

including many elements experienced by a ship at sea, the features come at a high cost. While 
prices on ruggedized computers have decreased significantly over the past two years, they 
currently cost three to four times as much as their non-ruggedized counterparts, with prices 
fi'om $5,000-$10,000. From a personal perspective, based upon five years of sea duty, COTS 
laptops survive remarkably well onboard a navy ship. The Army has deployed an expert 
system on both COTS laptops and ruggedized laptops. They found no major advantages for 
the ruggedized versions (Healfinan, 1996). The MAES project has always been an austerely 
fimded project. And while it is desirable to purchase both ruggedized and COTS portables 
for evaluation, at present, funding for ruggedized computers is not available. Therefore, the 
implementation plan calls for COTS laptop computers. 

3. Cost and Support 

Over the last decade the cost for computer hardware has constantly been declining. 
New generations of laptop computers are appearing approximately every six months. As a 
result, powerful laptops are fi-equently discounted 40-50% when a new generation appears. 
The aflfordable laptop computers available today for $1500 will provide ample power to run 
MAES efficiently. The least capable laptop CPU that is available at the time this thesis was 
written is a 486-DX4 75Mhz chip. Even these types of processors are becoming increasingly 
hard to find in quantity and by the time a purchase is made for deploying MAES, Pentium-75 
MHz chips will most likely be the cheapest CPU available. One of the lessons learned during 
the course of MAES development is that a low cost computer may not be the cheapest 
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alternative in the long run. Service support for generic brand notebooks has not been as good 
or as responsive as that of major name brand manufacturers. Reliability has also been a 
problem with generic brand laptops. With todays more liberated acquisition policy and 
numerous contracts or alternatives that meet competitive requirements, it is possible to 
purchase high quality computers with excellent warranty coverage. Indefinite Delivery, 
Indefinite Quantity (IDIQ) contracts. General Services Administration (GSA) schedules, or 
mail order companies are all available sources. 

H. SUMMARY 

In this chapter, the implementation issues discussed in Chapters n and m were applied 
to the initial implementation of MAES. Obtaining chain of command support for MAES was 
established as crudal to the success of MAES when deployed. Involving MK 92 technicians 
in the implementation and properly training all MAES users were also examined. The 
importance of supporting MAES users and conducting a user survey were addressed. F inally ^ 
hardware in^lementation issues were updated since they were last examined by John McGaha 
(1994). The next chapter provides a summary of conclusions and recommendations. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


Deployment issues addressed in this thesis apply to the implementation of the initial 
production version of MAES on the six Atlantic Fleet test frigates. This chapter summarizes 
the findings of this research. It also makes recommendations for fielding MAES and follow 
on research. 

A. SUMMARY OF CONCLUSIONS 

This thesis research was focused on one primary research question and four subsidiary 
research questions. The following discussion pertains to these research questions. 

1. What Are the Implementation Issues for the Initial Fielding of MAES? 

Two different efforts provide a keen inaght to implementation issues that confront the 
MAES deployment. According to Prerau (1990), implementing an expert ^stem is 
composed of three main phases; The initial phase, the core development phase and the final 
development and deployment phase. Each phase has certain tasks that must be completed 
before proceeding to the next phase. Following this implementation strategy ensures that an 
expert ^stem development team will consider and plan for applicable issues before the actual 
implementation begins. 

The Lucas-Ginzberg (1990) model of implementation examines implementation of a 
managanent information system (MIS) from the perspectives of man agemftnt and the users. 
Re\dew of this model provides insight to the implementation factors that can influence either 
management or user acceptance of a new MIS. Knowing which implementation factors will 
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affect management and users the most allows a deployment team to better prepare for the 
implementation. 

Drawing from the previous research efforts, the implementation factors that follow 
will prove to be critical for a successful implementation of MAES. 

• Obtaining management support. The MAES deployment team must include all 
levels of management in their training. It will be very important to gain then- 
support and keep it. 

• Involving users in the development. The MAES project team should continue to 
involve users and listen to their suggestions and concerns. 

• Properly training users. Professional training will be essential for users to use 
MAES. 

• Establishing a support infrastructure and effective lines of communication between 
the users and the maintenance team. Neglecting this area will turn many players 
against MAES. 

• Choosing the right hardware platform. A key reason for deploying MAES on 
notebook computers is to support the users need. Deploying the system on a 
desktop computer could potentially turn users away from using MAES if they 
ejq)erience delays getting access or if needed features are not av^able where the 
work is being done. 

• Conducting a post-implementation evaluation to determine user satisfaction and 
where improvements need to be made. This should be an ongoing process even 
after Fleet fielding. Continual product improvement should be ongoing. 

2. How Can Implementation Issues Concepts be Applied to the Initial 
Deployment of the MAES Initial Version? 

Applicable issues were discussed in great detail in Chapter V. The deployment issues 
that the MAES deployment team faces include: 

• Obtaining support from individual chains of command. Communications wU be 
essential for this issue. 
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• Involving MK 92 technicians in the implementation and further development of 
MAES should be ongoing. 

• Properly training all MAES users and their respective chains of command is 
essential. Deployment teams should have at least one senior enlisted sailor. 

• Assessing other alternatives to on-site training. The inconsistency of inport 
availability of FFG-7s may require one of the alternative methods of training. 

• Providing support for MAES hardware and software will be essential during the 
initial evaluation period. Slow or inconsiderate support could turn the community 
off on MAES. 

• Distributing and ensuring that users respond to a post-implementation survey will 
be essential for justifying deployment fleet wide. Their responses will be the most 
heavily weighted responses. 

• Purchasing hardware that balances value and cost and that proves to be reliable is 
important. Major hardware feilures that are not dealt with promptly, could result 
in an unfiivorable rating for MAES, even though the software performs admirably. 

3. How Can End-User Training Concepts be Applied to the Formulation of a 
MAES training plan for shipboard technicians? 

Chapter HI examined research that demonstrates the importance of properly training 
users v^en implemaiting expert systems. Users must not only be trained on how to use the 
system, but also how it works, why it is necessary, and what benefits it will provide. Properly 
assessing user’s basic training needs is the first step in developing a training plan. Other 
preliminary steps that need to be completed before actual training commences include: 

• Training the personnel who will be training the users. 

• Developing training material and methods. 

• Obtaining a preliminary evaluation of the training plan from involved parties. 
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The actual training that will be conducted should follow the training plan consolidated by the 
deployment team. Factors that need to be considered in the development of the training plan 
are: 

• The purpose and objectives of the training as they pertain specifically to MAES. 
MAES encounters the additional risks of introducing a new technology into the 
work environment. 

• The methods of training that will be used for each level in the chain of command. 

• The level of training appropriate for each level in the chain of command. Training 
should be tailored for each group. 

• The development of an evaluation form that will enable trainers to determine areas 
in which to improve. 

An effective training plan will enable the deployment team to provide sufficient, detailed 
information and hands on training to use MAES effectively and professionally. 

4. What are the Hardware Implemeiitatioii Issues Associated with Deploying 
MAES? 

The hardware issues associated with deploying MAES were addressed at the end of 
Chapter V. First a determination needs to be made on whether a notebook computer needs 
to be included as part of MAES. For the initial evaluation, it is recommended that MAES be 
pre-installed on a laptop and given to each test ship and to the FTSCLANT representatives. 
A laptop provides the technician with a degree of portability that is needed to effectively 
utili 2 e all the features ofMAES. A dedicated laptop also ensures that MAES will be available 
whenever the technician needs it to troubleshoot or train. A dedicated laptop also minimizes 
the conflicts that can arise with other software that may be installed on the computer and 
minimizes the introduction of viruses if users do not load other software programs. 
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A mggedized laptop is not presently affordable for evaluation. When purchasing the 
laptops, the least e3q)ensive new laptops from reputable brand name manufacturers should be 
purchased. Affordability of a minimally equipped system is not a problem. A laptop that has 
the minimum MAES hardware specifications is no longer even available in the marketplace 
unless it has been pre-owned. Therefore, an affordable model from a brand name that comes 
with an adequate warranty should be sufficient. The method of purchase that provides the 
best price should be used. 

5. What are the Lessons Learned in the Implementation of Other Expert 
Systems for Weapons Systems Within DOD? 

Two e:q)ert systems that were designed to diagnose weapons systems have recently 
been dq)loyed in the DOD. The activities that deployed these expert systems were contacted 
and the recommendations and lessons learned were discussed in Chapter IV. The key points 
include: 

• Training: a training team that visits each site was found to be valuable. Training 
must be coordinated with all involved commands. 

• Maintenance: useful suggestions are made by technicians after the systems were 
dq)loyed. E3q)ect to add more on-line help and features. 

• Post-implementation evaluation: technicians may not answer mailed surveys. 
Precautions to prevent unretumed surveys must be taken. 

• Hardware: technicians must have access to the system at all times. COTS laptops 
are durable enough to survive a maintenance environment but should not have a 
trackball pointing device due to its vulnerability to dirt and grease. 
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B. RECOMMENDATIONS 


The following subsections provide recommendations for deploying the initial version 
of MAES to the designated test ships and recommendation for future research in the 
implementation of MAES. 

1. Obtaining the Support of Upper Management 

Upper management support has been proven to be one of the strongest factors 
affecting an expert system’s successful implementation. Therefore, special attention needs 
to be paid to the effective presentation of the capabilities and benefits that MAES can provide 
to each ship. The presentation should highlight key findings in Powell’s (1993) cost benefit 
analysis to underscore the benefits available. A concise brief with a short demonstration 
should be made to each ship. A MAES advocate fi-om either FTC-SD, FCTC Dam Neck, or 
FTSC should accompany the deployment team to provide technical backing to the 
presentation. Management will look for sailor credibility and championing in order to fully 
support MAES themselves. 

2. Involving the MK 92 Technicians in the Implementation of MAES 

The opportunity for getting MK 92 technicians involved occurs during deployment 
training. During the training sessions the deployment team can highlight the contributions 
other technicians have made and the features in MAES that resulted. If resources are 
available, it is also recommended that a MAES newsletter be distributed periodically to inform 
all MK 92 technicians about their contributions. 
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3. Training MAES Users 

The training of MAES users should be conducted by a deployment team that travels 
to each training site. This method of training ensures that all MAES users on the test ships 
will receive sufiBdent quality training. Hands on exercises should be included in the training. 
The training plan provided as Appendix B may be used as a basis for the actual training. The 
training evaluation forms should be passed out at the conclusion of each training session and 
carefully read to determine if changes to the training plan need to be made. 

4. Providing Support to MAES Users 

A MAES newsletter and telephone support line should be available to aid users. The 
personnel that will be answering the phone will need to be knowledgeable in both the MK 92 
FCS and the MAES software. Additionally, a MAES world wide web page could be 
constructed in order to provide answers to frequently asked questions and provide 
downloadable software updates. Open communications and prompt, carefiil follow-up to 
each inquiry will be essential for maintaining user confidence in MAES support. 

5. Conducting and Analyzing the Results from the User Survey to Determine 
Fleet Suitability 

There will be no way to determine if MAES users are satisfied if a survey is not 
conducted. The methods discussed in Chapter in to increase user response to the survey 
should be used. Once the surveys are received, the data should be analyzed to determine user 
satisfaction and whether improvements are necessary. Continual testing of the system at the 
training centers can also provide more empirical evidence to compare with projected benefits. 
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6 . Utilizing COTS Hardware 

Because it is possible to get computers with three year warranties, and the relatively 
low cost, it is recommended that the initial issue MAES laptop be classified as a consumable 
item. If repair is beyond an economic price, the ship wiU be responsible for replacing it. All 
of this discussion is based on a premise that the initial evaluation will find it beneficial to 
deploy MAES on a notebook computer. 

Since the least capable laptop that is commercially available as of the writing of this 
thesis exceeds the MAES minimum hardware requirements, it is recommended that the 
MAES laptops be bought with cost as the primary consideration. The manufacturer should 
be a brand name that is reputable and well rated in current computer industry magayinpig 
while also providing a competitive warranty. The model that is purchased should also have 
at least a 10.4" color dual scan technology screen to ensure that help photos are displayed 
with sufficient resolution. 

NPS has evaluated the input pointing devices on notebook computes. The device 
should be integral to the computer. An attachable mouse/pointing device is not acceptable. 
Attaching and detaching will pose a reliability issue over time. It also may be lost or could 
more easily be broken than an integrated device. Trackball pointing devices tend to get dirty 
and are not appropriate. The touchpad pointing device has become popular on several 
computers. Its solid state design has proven reliable. However, users who used it did not like 
the “feel” or the saisitivity. Its effectiveness also may pose a problem if dirt and grease build 
up on it. The preferred device was the trackpoint stick first used in the IBM Thinkpad 
computers. 
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The minimum requirements for a MAES laptop are: 

• 486DX2 66MhzCPU 

• SMB RAM 

• 10.4" Dual Scan Color Screen 

• 340 MB Hard Drive 

• Integrated Trackpoint pointing device 

• 3.5" 1.44 MB floppy drive 
C. FOLLOW ON RESEARCH 

Currently, MAES is being evaluated by the MK 92 Mod 2 PCS instructor staff at 
FTC-SD. MAES is also planned to be evaluated on sk Atlantic Fleet ships in November. A 
survey should be distributed at least twice during the evaluation period. Follow on research 
could include an analysis of the data received fi-om the survey to determine whether MAES 
users are satisfied and what improvements need to be made. 
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APPENDIX A. IMPLEMENTATION PLAN FOR DEPLOYING THE INITIAL 

VERSION OF MAES 


Task Due Date 

Provide inputs to fleet scheduling conference to arrange test ships for 
deployment. 

Identify personnel for the deployment team. 

Establish a Deployment Team meeting schedule to coordinate 
preparations and make assignments. 

Establish a checklist to identify the problems, date and specific person to 
ensure all problems that come up are identified and solved. 

Ensure all documentation applicable to the deployment are available and 
complete. 

- Training Plan/Evaluation 

- Implementation Plan 

- User’s Manual 

Purchase required number of laptop computers. 

Send training plan to evaluating commands for review. 

Ensure that all members of the deployment team who will be conducting 
training are capable of teaching the required topics. 

Obtain training site fi’om local command in charge. 

Draft radio message informing participating commands of locations and 
times for training and briefs. 

Arrange lodging and transportation for the deployment team. 

Furnish required security clearances for deployment team access to 
applicable commands. 

Bmld PowerPoint slide show for individual command presentations (45 
min. duration). 

Deployment team reviews training plan. 

Configure laptops to optimally run MAES (install MAES, video settings, 
etc.) 
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Task 

Due Date 

Ensure that all required training materials are assembled (See training 
plan). 


Deploy MAES. 


Document lessons learned. 


Distribute user surveys. 
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APPENDIX B. MAES TRAINING PLAN 


Training will consist of two days at each site. The first day will consist of classroom training 
at a local training facility (FTC in Norfolk and base classroom in Mayport). The second day 
will consist of a brief visit to each ship to brief the chain of command and conduct follow-up 
training on MAES users. 

First Day 

Training session to commence at 0800 on the first day. 

Required materials: Overhead projector, training slides, student binders that contain copies 
of training slides and training evaluation forms, canned scenarios for MAES training, MAES 
HAV and S/W for users (one per ship), associated MAES documentation for users, and 
MAES laptop for instructor. 

Required attaidees: Ordnance OflBcer (first hour). Division Chief, Recommended that ships 
send all MK92 technicians. 

Training Syllabus: 

1. Background of MAES: 

A. What is an expert system 

B. The basics of an expert ^stem 

1) Components: Interface, Database, Inference Engine 

2) Expert: Knowledge acquisition, who he is 

3) The Technician: Most important part of the system 

4) MAES Capabilities 

a. What it can and cant do 

b. Discuss results of FTC-SD evaluation 

C. How MAES is different fi’om tech manuals 

1) 70% new knowledge 

2) Knowledge behind MAES is fi-om a UNISYS engineer with 30 years 
experience 

D. Why the system is important 

1) It WU Save You Time! 

2) It will save money! 
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2. Intro to MAES 


A. Hardware 

1) Proper care 

2) Basic operation 

3) Custody forms 

4) Repair procedures 

- Who to contact for problems with the S/W or HAV 


B. Software 

1) Installation 

2) User's Manual contents 

LUNCH 
3. MAES Operation 

A. Navigating the program 

1) Opening the program 

2) Going from screen to screen 

B. Canned Scenarios 

1) Go through scenarios step-by-step 
- One easy and one hard 

C. Fill out training evaluation forms 
Second Dav 

Will consist of a ship visit to each ship that MAES will be evaluated on. A space with an 
overhead to brief the chain of command on each ship will need to be pre-arranged. The dates 
and times of these briefs will have to be forwarded to the ships as early as possible so they can 
schedule them. 

1. Short (1/2 hr) brief to chain of command (CO,XO,CSO) 

OUR CHANCE TO GET MANAGERIAL SUPPORT FOR MAES 

A. What MAES does 

B. How it will save the sailor time, make his job easier and save the ship money 

C. Display Cost/Benefit analysis findings 
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D. Convince the command of the importance of using the system 

- As part of the evaluation they are extremely important as they will be 
putting the system through its paces 

2. Brief follow on visit with the technicians to see if they have any last minute questions. 
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TRAINING EVALUATION FORM 


This evaluation form is provided for you to fill out in order that we can provide you with 
high quality trmning. We ask for your cooperation in filling out this short questionnaire. 
It should take no more than five minutes of your time. Providing us with your name is 
optional only. Please turn the completed questionnaire in to a member of the MAES 
deployment team when you are done. Thank you very much for your assistance. 

Instructions: Please circle the appropriate response and provide comments, if any, in the 
space provided at the end of the questionnaire. 

1. Was your training room comfortable? 

Yes No 

2. Did the instructor tell you what the training objectives were? 

Yes No 

3. Do you feel you have an imderstanding of what MAES can and cannot do? 

Yes No 

4. Do you understand how to install MAES fi-om the floppy disks? 

Yes No 

5. Do you know who to contact in the event you have a problem or question about 
MAES? 

Yes No 

6. Do you understand how to troubleshoot with MAES? 

Yes No 

Comments: 
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APPENDIX C. MAES USER’S MANUAL 


MK 92 MOD 2 FCS MAINTENANCE ADVISOR 
EXPERT SYSTEM (MAES) 



USER’S MANUAL 
VERSION 1.0 
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Chapter 1 


Welcome to MAES 


Introducing MAES 

The MK 92 Mod 2 Maintenance Advisor Expert System (MAES) was developed to assist fire 
control technicians in the fleet to correctly diagnose casualties in the MK 92 Mod 2 Fire 
Control System. MAES will increase your operational readiness by giving you “expert” 
knowledge when and where you need it most. MAES will assist in locating faults discovered 
in radar system Performance or Calibration during the Daily Systems Operability Test 
(DSOT). The capability to diagnose RF Power Checks will be included in a later version. 

MAES was designed with the following goals in mind: 

• Improving your workload by decreasing the amoimt of time you spend diagnosing 
faults in the system. 

• When required, the ability to communicate with shore based maintenance 
organizations (such as FTSC) more eflFectively concerning problems encountered. 

• Provide you with a training tool for brushing up your troubleshooting skills. 

• Inaeasing ship’s state of operational readiness by decreasing the amount of time 
required to repair the MK 92 FCS. 

• Eliminating circuit cards that are turned in that have No Fault Evident (NFE). 

• Saving ships money through reduced ordering of repair parts. 

The knowledge that is behind the power of MAES comes fi'om senior engineers with many 
years experience in diagnosing the MK 92 Mod 2 FCS. 


Note: This manual assumes you are familiar with your computer’s basic operations. If you 
are unfamiliar with terms like “dialog” or “double-click”, or basic file handling and 
printing procedures, please read the owner’s manual for your computer and your 
Microsoft Wirukfws/Windows 95 User’s Guide before using MAES._ 
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Chapter 2 


Installation 


Reviewing Your Package Contents 

Before installing MAES, please take a moment to verify that your package contains the 
appropriate items and that your computer meets the system requirements for MAES. Your 
MAES package should contain the following items: 

Two MAES Installation disks, 3.5" 1.44 MB 

PCS MK92 Maintenance Advisor Expert System User’s Manual 

If this is your initial receipt of MAES you should also be in receipt of a laptop computer. 
Please refer to the documentation that came with the computer for it’s care and maintenance. 

If any items are missing, please contact MAES HelpLine immediately at (805) 982-0141 or 
DSN 551-0141. 

Checking System Requirements 

MAES disks are not bootable; they do not contain MS-Windows software. To operate 
MAES, you need the following hardware and software. 

MAES is operational tmder Windows 3.xx or Windows 95. The system requirements for 
MAES follow: 

• IBM PC or 100% compatible 

• Intel 80386 DX or higher (486 or higher recommended) 

• hftcrosoft Windows 3. lx or Windows 95 

• A hard disk drive with at least 30 MB of free storage space 

• Minimum 4 MB RAM (8 MB RAM recommended) 

• A 3.5 inch 1.44 MB floppy drive 

_ • A standard VGA monitor that supports 640 x 480 resolution and 256 colors 

Note: Less capable monitors may be used, but resolution quality may be compromised. If 
you have a monitor that is 17" or larger, or a graphics card with more than 2 MB of 
_ video RAM, you may achieve better results with a resolution of 800x600. _ 

• Mouse or built-in pointing device 
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Installing MAES 


Installing MAES 


Installation takes about fifteen minutes. 

Before you install MAES, please: 

• Close other applications 

Closing applications (such as a word processor or spreadsheet program) will fi’ee 
memory and prevent possible conflicts during installation. 

• Turn off virus protection programs 

Some virus protection programs can interfere with the installation process. Please 
refer to the user’s manual for your virus protection software for information on 
how to turn off your virus protection program. We also recommend turning off 
any installed memory-resident programs (such as a screen saver or a pop-up 
program that appears when you type certain key combinations). 

• Make a backup of your MAES program disks 

See Making a Backup of your MAES Disks at the end of this chapter for more 
information. 

Once you have closed all other ^^^dows applications, disabled virus protection, and made 
a backup of your MAES program disks, you are ready to install MAES. Follow these simple 
steps to install MAES. 

Note: The following instructions assume your hard disk is drive C and your floppy disk is 
drive A. If named differently, please use your drive designators. 

1. Insert the installation disk 1 of 2 in a 3.5" drive. 

For Windows 3.1x users: 

2. Choose Run firom the Program Manager File menu. 

3. Type the command AnsrSTALIT.EXE and click OK. Go to step 8. 

For Windows 95 users: 

2. From your desktop, double-click the My Computer icon. 

3. Double-click the Control Panel icon. 

4. Double-click the Add/Remove Programs icon. 

5. In the Add/Remove Programs Properties window, choose the Install/Uninstall tab. Click 

Install. 

6. Click Next. 

7. The Command line in the installation window should read A:\TNST AT .tt f.yf click 
Finish. Go to step 8. 
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Installing MAES 


Installing MAES (cont) 

8. The install program will display a panel telling you that it is copying the install program 
to a temporary area on your hard drive. 

9. The install program will display a panel telling you about itself. 

10. The install program will begin displaying windows prompting you for responses. 

11. When you are prompted to Select Option, select Install MAES 1.0 and click on OK. 

12. When you are prompted to Select hnstallation Drive select a drive from those listed that 
has at least 30 MB free space. 

13. You will be prompted to specify an installation path. The recommended installation 
directory is named MAES 10. The recommended installation drive is where Windows 
is installed. For example, if you have Windows installed on drive C, C:\MAES10 is the 
suggested installation directory. Enter a different path if you desire and then press OK. 

Warning: Do not specify the \Windows or \Windows\System directories as the MAES 

directory. However, C:\Windows\MAES10 is acceptable. 

14. At this point the install program will begin copying files to your hard drive. A progress 
meter will show you the status of the installation throughout this process and will prompt 

you for a disk change. 

15. When prompted to Install Icon, select the second option Install Icon in a New Group 
and press OK unless you prefer to place the icon in an existing group. 

16. The Install MAES 1.0 menu will reappear. Select Exit Installation and click OK. 

17. Windows 3.lx users should find the new icon MAES 1.0 in the MAES 1.0 program 
group and can run it by double-clicking on it. Windows 95 users can run MAES by 
clicking on Start - Programs - MAES 1.0 - MAES 1.0. 

18. Remember, at this time only the Calibration and Performance options are available for 
use. 


MAES is now ready to use! 



Installing MAES 


Making a Backup of Your MAES Disks 

To make a backup of your MAES program disks, simply follow these steps: 


Note: The source and destination disks must be of the same size (3.5") and capacity 
(1.44 MB) _ 

Windows 3. lx users: 

1. Insert the source disk (MAES program disk). 

2. Open the file manager and click on Disk. 

3. From the drop down menu, select Copy Disk. 

4. When prompted, insert a blank disk to copy onto. 

5. Repeat steps 1-4 for subsequent MAES disks. 

Windows 95 users: 

1. Open My Computer. 

2. Click the icon for the disk you want to copy (A: or B:) 

3. Select File [ Copy Disk fi-om the My Computer menu bar. The copy Disk dialog will be 
displayed. 

4. Select the Copy fi-om (source) drive and the Copy to (destination) drive. These drives can 
be the same. For example, both can identify the A: drive. 

5. Choose the Start button. Follow the on-screen instructions. 
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Chapter 3 


MAES Basics _ 

MAES is extremely flexible and powerful. The interface was designed to be as simple as 
possible so you can access the knowledge quickly and effectively. This chapter introduces 
the basic prindples and procedures you need to understand to work effectively and efficiently. 


Screen Layout _ 

The standard MAES screen is divided into three primary sections. These sections, as shown 
in Figure 3-1, are the title bar, procedure area, and action area. 



Figure 3-1. MAES Standard Screen Layout 

Title Bar 


The MAES title bar is always positioned horizontally across the top of the screen. The title 
bar displays information about where you are in the MAES program. For example, the title 
bar in Figure 3-1 says “All CAS Track Modes FF and FA” and below it “Power High 
Occurs”. This indicates that you are troubleshooting a problem with the CAS Track Modes 
FF and FA and are currently determining whether a Power High condition exists. The title 
bar will change as you move from one area of the program to another. 
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MAES Basics 


Procedure Area 

The Procedure Area is the central portion of the MAES display. In Figure 3-1 it is the 
horizontal bar labeled “Procedure”. MAES uses this area to display instructions, procedures, 
and questions. 

Action Area 

This area is located in the bottom area of the screen. In Figure 3-1 it is the horizontal bar 
labeled “Action”. This is the area that allows you to interact with MAES by clicking on 
buttons. In Figure 3-1, the buttons that are available are “Yes” and “No”. 

Screen Identification Number (SIN) 

At the lower, right-hand comer of every MAES screen there will be a small number. For 
example, in Figure 3-1 it is “13". Refer to this number and the Title Bar if you have questions 
about MAES or are submitting a Software Feedback Form. 


Input Methods _ 

The MAES display screens provide a variety of methods to interact with the program. The 
most common methods are buttons, list boxes, and check boxes. These objects are used to 
teU MAES what you want it to do in the way of displaying results, instmctions, or help. 

Buttons 

Inputs to MAES are performed by clicking on buttons. For example, when the button labeled 
“Prev Screen” in Figure 3-1 is pressed, MAES will display the previous screen. Buttons 
appear on almost all of the MAES screens. You can only select one button at a time. 
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MAES Basics 



Figure 3-2. MAES Screen Using List Box. 


List Boxes 

List Boxes are another way to interact with MAES. List Boxes display a list of choices that 
you may select one option from. If the list box has more choices than can be displayed in the 
box, then you can use the scroll bars to view all of the contents. Only one item in a list box 
can be selected at a time. Figure 3-2 is an example of a MAES list box. 



































MAES Basics 



Figure 3-3. MAES Screen Using Check Boxes 


Check Boxes 

A check box allows you to select or clear an option. You can select as many options as are 
available by checking their corresponding check boxes. A is placed in a check box when 
it has been selected. An example of check boxes in MAES is shown in Figure 3-3. Notice 
that six check boxes in the left column have been selected. 
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MAES Basics 



Figure 3-4. MAES Screen with Help Buttons 


Help Information 

There are three types of help information available to you in MAES. These types include 
“How”, “Why”, and “Parts Info”. Please refer to Figure 3-4 during the discussion on the help 
features. 


How 

Clicking on a “How” button gives you a detailed description of how to perform a required 
procedure. Once you press a button labeled “How” in the procedure area, a new screen 
appears with the information you requested. If there is more information than can be 
displayed in one screen, it will appear in a scrollable window. 
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MAES Basics 


Photo 

At the bottom of some “How” screens, this button may be available. Clicking on the “Photo” 
button gives you a photo or series of photos that step you through a procedure that you may 
need more inJformation on. These photos will appear best if your monitor is set for 640x480 
resolution with 256 colors. You may always click on the “Return” button to go back to the 
Procedure screen where you left off. 


Why 

Clicking on a “Why” button provides you with an explanation of why MAES is directing you 
to perform a certain task. This “Why” information gives you insight into how the MK 92 
expert we used to design MAES attacks a problem. Once you finish reading the explanation, 
press “Return” and MAES wUl take you back to the previous screen. 


Parts Info 

By clicking on a “Parts Info” button MAES will give you parts information with regards to 
the part that MAES is recommending you replace. The parts information will provide you 
with a part number, reference diagram number, if and where the part is used elsewhere in the 
system, part designation, and the power requirement of that part. 

Help Buttons 

Clicking on “Help” buttons within MAES will provide you with specific help about the screen 
you are currently viewing. 
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MAES Basics 




CA& Track Target Wodes 

OkM^ul FiMvd£titHLStt.Tol«ttttAi:^ 









^ Generic Circuit Cerd Replacement: 


ai- Removeandinspectcardteards far evidence of damage. 

12. Replace damaged cardJcards. (See Paris Inform^on page.) 


i; 




3. If inspection was o. k. reinstall card/cards and recheck previous test 
(Ensure cardteards are seated property.) 

I. If test still fails, replace cardfcards one at a dme beginning with the first 
card Ijsted (most aptto faifi and rccheck test 

S. When available, use identical cardfcards from other locations as test 
replacement cards. (See Pate Information page.) 

5. fftestsfill falls after all cards have been replaced, check circuit associated 
v«th cards, (See Pate Infbrmafion page.) 




1 

I 

i 


if^ 

"""'i 



Figure 3-5. Circuit Card Replacement Screen. 


Circuit Card Replacement 

CAUTION: It is very important to perform the generic part replacement methodology 
when replacing parts. 

Most drcuit cards in the MK 92 Mod 2 FCS can be replaced using a generic card replacement 
procedure. Certain rules, which are intuitive to Fire Control Technicians, must be followed 
when repladng a circuit card. Procedures for generic card replacement procedure and rules 
to follow after card replacement are available in MAES as depicted in Figure 3-5. 
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MAES Basics 


Rules for Exiting MAES After Part Replacement 

The rules that should be followed after a part is replaced are as follows: 

1. If an adjustment is performed in the path where the part is to be replaced, exit and perform 
the adjustment again. 

a. If the adjustment can be performed within specifications, rerun DSOT. 

b. If the adjustment still can not be performed within specifications, return to the 
display screen for part replacement and replace the next part listed. Perform the 
adjustment again. 

c. If all parts have been replaced and adjustment is still out of specifications, return 
to the part replacement screen. Obtain the figure reference for the signal flow 
diagram associated with parts and continue with troubleshooting. 

2. If an adjustment is not performed in the path where the part is replaced, exit to the 
submenu display screen and check to see if the problem is corrected. 

a. If the problem is corrected, rerun DSOT. 

b. If the problem is not corrected, return to the part replacement screen and replace 
the next part listed. Check and see if the problem is corrected. 

c. If all parts have been replaced and the problem stUl exists, return to the part 
replacement screen. Obtain the figure reference for the agnal flow diagram associated 
with the part and continue troubleshooting. 


Note: When returning to the adjustment screen or submenu display screens, make certain 
that the initial setup is completed before performing an adjustment or when checking 
_ to see if the problem is corrected. _ 
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Chapter 4 



The following sample session is an example of how to use MAES for troubleshooting a fault. 
The example assumes a failure in the Calibration module which has failure in all CAS Track 
modes. 

Begin MAES 

Start MAES by following the steps previously discussed at the end of the Installing MAES 
section. To begin MAES, click on the button labeled ‘TBegin Program” on the opening screen 
with the picture of a FFG-7 class ship. 

Select A Module 


The next screen that appears will be the MAES Main Menu and is shown below in Figure 4-1. 
Since this example is assuming a Calibration CAS Track failure, press the Calibration button 
on the left side of the Action area. 



Figure 4-1. MAES Main Menu 
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Figure 4-2. Calibration Menu 

Once the Calibration button is pressed, the Calibration Menu that is depicted in Figure 4-2 
will be displayed. 


Select DSOT Entry Method 

Two options are available for entering DSOT data on the Calibration Menu: Printout and 
Manual. The Printout option allows you to select problem areas on a display screen, shown 
in Figure 4-3, that mimi c a DSOT printout. Click on the button labeled “Printout” 
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Figure 4-3. Calibration DSOT Printout Menu 


Select Failed Areas 

The next step is to select the failed areas by using check boxes in the Printout Menu. Check 
the six boxes shown in Figure 4-3 that are labeled CAS TR TGT FF, CAS TR CLT FF, CAS 
TR ECM FF, CAS TR TGT FA, CAS TR CLT FA, and CAS TR ECM FA. Make certain 
that a appears in each box. Checking these boxes informs MAES that multiple CAS 
Track Mures exist. Once the check boxes have been selected, press the “Continue” button 
on the Cahbration DSOT Printout Menu. 

An ordered list of paths, in this case only one that says “All CAS Track Modes”, that MAES 
will take you through will be displayed. Click on the “Continue” button to start performing 
troubleshooting tasks. 
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Ail CAS TtBCk Moi&S FP and PA 





Figure 4-4. Power Hi Task Screen 


Perform Tasks 

The next screen is shown in Figure 4-4. MAES asks if a power HI condition exists for all 
track modes, so click on the “Yes” button in the action area. This action causes MAES to 
display another task screen as shown in Figure 4-5. 
































































































Figure 4-5. Measure TTL Levels 


Completing a Task 

The screen shown in Figure 4-5 instructs you to measure TTL Remote Mode Logic Levels 
at UD412/A1A5-A13. If you require help on how (or why) to perform this procedure, you 
can request on-line help as explained in the next section. 


Getting Help 

Press the button labeled “How” as shown in Figure 4-5. MAES will give you detailed 
instructions on how to measure TTL Remote Mode Logic Levels at UD412/A1A5-A13. A 
view of this information is shown in Figure 4-6. Use the scroll bars to read the entire set of 
instructions. 
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. AJI CAS Track FF and FA. :- 

s Power Higfc Cicwrs: 




11 .Plac9 DSOTtests«t(UD412) in REMOTE by perfroming the following: 

I ^ AtUD461,pre$sTOTC $EtJECTN MISOTEST, MINUS LEFTJDRQP 
I DOWN, 03 and INSBIT pushbuttons 

I b}AtUD4T2,v«rifyREMOTEtampislit 

P 2. Waftuntil calibration iscompl^ett befbretaking any measurements 
^ (approximate^ ^ minutes). 

1 

13. AtUD412iA1AS-A13 cjieck logic levels at TP15, TP17, andTP22 
i tisingantultmieter. 

I 

14. Record results. 

p S. Return to MANUAL mode wdien test is completed. 












Figure 4-6. HelpScreen (How to Measure TTL Remote Mode Logic Levels) 


Once you have finished reading the help information, click on the button labeled “Return”. 
MAES will return you to the previous screen (Figure 4-5). Now click on the button labeled 
“Why”. MAES gives you a concise reason why you are instructed to perform this 
measurement. The Why screen is depicted in Figure 4-7. After reading the steps in the Why 
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display, exit back to the Measure TTL Levels by clicking on the “Return” button. 

Since an all CAS Track Failure in the example indicates that a logic level at TP 15, TP 17, or 
TP22 is high, press the button labeled “Yes” in Figure 4-5. This action will cause MAES to 



Figure 4-7. Replace Parts Screen 
display the Replace Parts Screen as shown in Figure 4-7. 

Replace Failed Part 

When replacing parts you may access part information that would otherwise take you hours 
to access via normal supply channels and technical manuals. To use this information, click 
on the button labeled “Parts Info”. The type of information that is available is displayed in 
Figure 4-8. 
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Figure 4-8. Parts Information Screen 


Exiting MAES 

After you have obtained the part information you need to replace a part, click on the “Return” 
button. You are now back on the Replace Parts Screen. Clicking on the button labeled 
“Continue” will return you to the Calibration Main Menu. You may now exit MAES. This 
concludes the sample session. 
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Chapter 5 


Documentation and Technical Support _ 

Technical support regarding MAES may be received from representatives at NAVSEA by 
mail^ telephone, or by GENADMIN message. 

NAVSEA Assistance 

Hardware and software technical support may be obtained from the NAVSEA technical 
representatives at the Naval Surfece Warfere Center, Port Hueneme Division. The NAVSEA 
point of contact is Mr. Henry Seto, and he can be reached at: 

Mr. Henry Seto (Code 4C46) 

Port Hueneme Division, Naval Surface Warfare Center 
4363 Missile Way 
Bldg. 1211 

Port Hueneme, CA 93043-4307 
(805) 982-0141 commercial or 

551-0141 DSN, e-mail: SETO_HENRY@OM.NSWSES.NAVY.MIL 
Message Address: RUWFPBC/NAVSURFWARCENDIV PORT HUENEME CA//4C00// 

Documenting Problems and Suggestions 

We believe that one of the reasons MAES is so effective at solving faults in the MK 92 PCS 
is that you, the fleet technician, have been a key component in the development of this 
software. It was people like you viio suggested that we include photos and parts information 
which have become key features of MAES. If you have a suggestion that you feel would 
make MAES a more effective tool, please fill out the Digital Systems Feedback Report 
included as the last page of this manual. You may also use the same form to report a 
recurrent problem that may be occurring with your software or computer. 


117 


















APPENDIX D. MAES USER SURVEY 

1. What is your current rank? 

□ El □ E2 □ E3 □ E4 □ E5 □ E6 □ E7-E9 

2. How many years of experience have you had as a MK92 tech? 

□ Less than 1 year 

□ 1-2 years 

□ 3-4 years 

□ 5-6 years . 

□ 7-8 years 

□ 9 or more years 

3. On what ship are you currently stationed on? _ 

4. What is your current billet? 

□ Technician 

□ Work Center Supervisor 

□ Divisional LPO 

□ Divisional CPO 

□ Other 

5. When you had a feult to troubleshoot, what role did MAES play? 

□ Did not use MAES to troubleshoot 

□ Used MAES in conjunction with the technical manuals □ Used MAES only 

6. Have you used MAES for (check all that apply): 

□ Troubleshooting 

□ Training 

7. How often do you use MAES? 

□ Never 

□ For all maintenance troubleshooting covered by MAES 

□ For part of maintenance troubleshooting covered by MAES 
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8. How has MAES, in comparison to only using technical manuals for troubleshooting, 
changed the time required for you to repair a feult? 

□ Could not have solved some faults without MAES 

□ Significantly less amount of time to repair faults 

□ Less amount of time to repair faults 

□ More time is required to repair faults 

□ Significantly more time is required to repair faults 

Please rate MAES in the following areas: 


7. Ease of use 

Veiy 

Dissatisfied 

1 2 3 

4 

Very 

Satisfied 

5 

8. Ability of MAES to solve faults 

1 2 

3 

4 

5 

9. MAES User Manual 

1 2 

3 

4 

5 

10. Viewscreen display (sharpness/brightness) 

1 2 

3 

4 

5 

11. Supply information 

1 2 

3 

4 

5 

12. Technical photos 

1 2 

3 

4 

5 

13. Explanations of troubleshooting steps (Hows) 

1 2 

3 

4 

5 

14. Explanations of troubleshooting 
methods (Whys) 

1 2 

3 

4 

5 

15. Online help 

1 2 

3 

4 

5 

16. Your confidence in the system 

1 2 

3 

4 

5 

17. Reliability of: 

Laptop Computer 

1 2 

3 

4 

5 

MAES Software 

1 2 

3 

4 

5 

18. Your overall satisfaction with MAES 

1 2 

3 

4 

5 
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Comments: 
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